Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-15 Thread Florian Pflug
On Apr14, 2013, at 17:56 , Fujii Masao masao.fu...@gmail.com wrote: At fast shutdown, after walsender sends the checkpoint record and closes the replication connection, walreceiver can detect the close of connection before receiving all WAL records. This means that, even if walsender sends all

Re: [HACKERS] WIP: index support for regexp search

2013-04-15 Thread Alexander Korotkov
I found you committed GiST index implementation. That's cool. I found an easy way to optimize it. We can also use trigramsMatchGraph for signatures. Attached patch contains implementation. Simple example in order to demonstrate it: Before the patch: test=# explain (analyze, buffers) select *

[HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that wouldn't cause problems: any volatile function that

[HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread hubert depesz lubaczewski
In current 9.3, I see: $ select p.proname, p.provolatile from pg_proc p join pg_namespace n on p.pronamespace = n.oid where n.nspname = 'pg_catalog' and p.proname ~ 'json'; proname | provolatile ---+- json_in | s

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 03:00:34PM +0100, Simon Riggs wrote: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile

Re: [HACKERS] WIP: index support for regexp search

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 05:53:41PM +0400, Alexander Korotkov wrote: I found you committed GiST index implementation. That's cool. I found an easy way to optimize it. We can also use trigramsMatchGraph for signatures. Attached patch contains implementation. Simple example in order to

Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Andrew Dunstan
On 04/15/2013 11:16 AM, hubert depesz lubaczewski wrote: In current 9.3, I see: $ select p.proname, p.provolatile from pg_proc p join pg_namespace n on p.pronamespace = n.oid where n.nspname = 'pg_catalog' and p.proname ~ 'json'; proname | provolatile

Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread hubert depesz lubaczewski
On Mon, Apr 15, 2013 at 11:31:39AM -0400, Andrew Dunstan wrote: Me either. It's an oversight, really. Unless there is any objection I'll change them toot sweet. What about the existing (as of 9.2) functions? I don't think that 9.2 functions are that interesting, since these are to build json

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Heikki Linnakangas
On 15.04.2013 17:00, Simon Riggs wrote: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that wouldn't

Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Is there any particular reason extract functions (object_field/array_element/...) can't be immutable? I can't readily imagine a situation where output of these functions would change for different queries. Me either. It's an oversight, really.

Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Andres Freund
On 2013-04-15 11:31:39 -0400, Andrew Dunstan wrote: On 04/15/2013 11:16 AM, hubert depesz lubaczewski wrote: In current 9.3, I see: $ select p.proname, p.provolatile from pg_proc p join pg_namespace n on p.pronamespace = n.oid where n.nspname = 'pg_catalog' and p.proname ~ 'json';

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: I claim this is a common class, since sequence next_val functions and uuid generators meet that criteria and most common forms of auditing trigger, as well as any other form of data-reformatting trigger. Since this is a common case,

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:41, Heikki Linnakangas hlinnakan...@vmware.com wrote: What I'd like to do is to invent a new form of labelling that allows us to understand that COPY can still be optimised. It would be even nicer to detect at runtime, when a default expression or before trigger tries to

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other words, how do we know what resources (time, CPU cycles, disk seeks, etc.) are being consumed here? The multi-insert

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other words, how do we know what resources (time, CPU cycles, disk seeks,

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 11:49:42AM -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote: On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:08, David Fetter da...@fetter.org wrote: Loading data into a table with a SERIAL or UUID column is the main use case, so I'll measure that. The former is common enough a use case to optimize specifically, should the numbers come out right. Do you suppose that an in-core

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Hannu Krosing
On 04/15/2013 06:04 PM, Simon Riggs wrote: On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other words, how do we know

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
David Fetter da...@fetter.org writes: On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote: Loading data into a table with a SERIAL or UUID column is the main use case, so I'll measure that. The former is common enough a use case to optimize specifically, should the numbers come out

Re: [HACKERS] WIP: index support for regexp search

2013-04-15 Thread Tom Lane
Alexander Korotkov aekorot...@gmail.com writes: I found you committed GiST index implementation. That's cool. I found an easy way to optimize it. We can also use trigramsMatchGraph for signatures. Attached patch contains implementation. Good idea, committed. regards,

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:04, Simon Riggs si...@2ndquadrant.com wrote: I will implement as a kluge, test and report the results. Test is COPY 1 million rows on a table with 2 columns, both bigint. Verified no checkpoints triggered during load. No other work active on database, tests condicted on

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 06:30:55PM +0100, Simon Riggs wrote: On 15 April 2013 17:04, Simon Riggs si...@2ndquadrant.com wrote: I will implement as a kluge, test and report the results. Test is COPY 1 million rows on a table with 2 columns, both bigint. Verified no checkpoints triggered

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 18:41, David Fetter da...@fetter.org wrote: The difference between HEAD and patch in the COPY, with sequence case is pretty remarkable. What's the patch? Attached. This is usable only for this test. It is not anywhere remotely close to being applied. -- Simon Riggs

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 07:04:55PM +0100, Simon Riggs wrote: On 15 April 2013 18:41, David Fetter da...@fetter.org wrote: The difference between HEAD and patch in the COPY, with sequence case is pretty remarkable. What's the patch? Attached. Thanks! :) This is usable only for this

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: I claim this is a common class, since sequence next_val functions and uuid generators meet that criteria and most common forms of auditing trigger, as well as any other form of data-reformatting trigger. I don't believe that

[HACKERS] Query not using index for user defined type

2013-04-15 Thread Rodrigo Barboza
Hi guys. I created a type 'mytype' (an unsigned int) and created an operator class for index. Then I created a table with a column of my type and isnerted 1000 entries. But no matter how many entries I have in the table, it never uses the index. It always does a seq scan. Here is the explain

Re: [HACKERS] [PATCH] pg_regress and non-default unix socket path

2013-04-15 Thread Robert Haas
On Fri, Apr 12, 2013 at 1:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The hunk that changes the messages might need some thought so that it doesn't cause a translation regression. But in general I see no reason not to do this before we release beta1.

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: I don't believe that it's a good idea to consider nextval() to be reorderable, so I'm not convinced by your argument here. Why not? I admit that I can't convince myself that it's

Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Jaime Casanova
On Mon, Apr 15, 2013 at 3:08 PM, Rodrigo Barboza rodrigombu...@gmail.com wrote: Here is the explain analyze with 1000 entries: explain analyze select * from mytable where a 120::mytype and a 530::mytype; I'm not sure this is appropiate for -hackers, maybe should post on -general. Also

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Jaime Casanova
On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: OTOH, the notion that a UUID generator doesn't touch *any* database state seems like it might be worth treating as a general function property: it's simple to understand and applies to a lot of other volatile functions such

Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Tom Lane
Rodrigo Barboza rodrigombu...@gmail.com writes: I created a type 'mytype' (an unsigned int) and created an operator class for index. Then I created a table with a column of my type and isnerted 1000 entries. But no matter how many entries I have in the table, it never uses the index. It

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 20:52, Robert Haas robertmh...@gmail.com wrote: On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: I claim this is a common class, since sequence next_val functions and uuid generators meet that criteria and most common forms of auditing trigger, as well as

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 4:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think plenty of people would be upset if row serial numbers assigned with nextval() were not assigned in the order of the incoming rows. The argument that you can get gaps in the sequence in some corner cases (none of which

Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Andrew Dunstan
On 04/15/2013 11:46 AM, Andres Freund wrote: Me either. It's an oversight, really. Unless there is any objection I'll change them toot sweet. What about the existing (as of 9.2) functions? ISTM json_in, out, recv, send should also be immutable. array_to_json, row_to_json et all can't be tho.

Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Rodrigo Barboza
On Mon, Apr 15, 2013 at 5:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Rodrigo Barboza rodrigombu...@gmail.com writes: I created a type 'mytype' (an unsigned int) and created an operator class for index. Then I created a table with a column of my type and isnerted 1000 entries. But no

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 21:32, Jaime Casanova ja...@2ndquadrant.com wrote: On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: OTOH, the notion that a UUID generator doesn't touch *any* database state seems like it might be worth treating as a general function property: it's simple

Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread anara...@anarazel.de
Andrew Dunstan and...@dunslane.net schrieb: On 04/15/2013 11:46 AM, Andres Freund wrote: Me either. It's an oversight, really. Unless there is any objection I'll change them toot sweet. What about the existing (as of 9.2) functions? ISTM json_in, out, recv, send should also be immutable.

Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Tom Lane
Rodrigo Barboza rodrigombu...@gmail.com writes: I created a implic cast for mytype to bigint. So when I do the same query it does seq scan, because the column is transformed into bigint. Yeah. One reason why there's not an unsigned int type already is that it seems impossible to shoehorn it

Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Rodrigo Barboza
On Mon, Apr 15, 2013 at 7:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Rodrigo Barboza rodrigombu...@gmail.com writes: I created a implic cast for mytype to bigint. So when I do the same query it does seq scan, because the column is transformed into bigint. Yeah. One reason why there's not

Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Bruce Momjian
On Mon, Apr 15, 2013 at 04:41:53PM -0400, Andrew Dunstan wrote: On 04/15/2013 11:46 AM, Andres Freund wrote: Me either. It's an oversight, really. Unless there is any objection I'll change them toot sweet. What about the existing (as of 9.2) functions? ISTM json_in, out, recv, send should

[HACKERS] event trigger API documentation?

2013-04-15 Thread Peter Eisentraut
I'm having trouble finding documentation about how to write event triggers. The chapter in the documentation http://www.postgresql.org/docs/devel/static/event-triggers.html says they can be written in C or supported PLs, but does not explain it any further. Is there any documentation for it?