Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-15 Thread Florian Pflug
On Apr14, 2013, at 17:56 , Fujii Masao masao.fu...@gmail.com wrote:
 At fast shutdown, after walsender sends the checkpoint record and
 closes the replication connection, walreceiver can detect the close
 of connection before receiving all WAL records. This means that,
 even if walsender sends all WAL records, walreceiver cannot always
 receive all of them.

That sounds like a bug in walreceiver to me.

The following code in walreceiver's main loop looks suspicious:

  /*
   * Process the received data, and any subsequent data we
   * can read without blocking.
   */
  for (;;)
  {
if (len  0)
{
  /* Something was received from master, so reset timeout */
  ...
  XLogWalRcvProcessMsg(buf[0], buf[1], len - 1);
}
else if (len == 0)
  break;
else if (len  0)
{
  ereport(LOG,
  (errmsg(replication terminated by primary server),
   errdetail(End of WAL reached on timeline %u at %X/%X,
 startpointTLI,
 (uint32) (LogstreamResult.Write  32),
 (uint32) LogstreamResult.Write)));
  ...
}
len = walrcv_receive(0, buf);
  }

  /* Let the master know that we received some data. */
  XLogWalRcvSendReply(false, false);

  /*
   * If we've written some records, flush them to disk and
   * let the startup process and primary server know about
   * them.
   */   
  XLogWalRcvFlush(false);

The loop at the top looks fine - it specifically avoids throwing
an error on EOF. But the code then proceeds to XLogWalRcvSendReply()
which doesn't seem to have the same smarts - it simply does

  if (PQputCopyData(streamConn, buffer, nbytes) = 0 ||  
  PQflush(streamConn))
  ereport(ERROR,
  (errmsg(could not send data to WAL stream: %s,
  PQerrorMessage(streamConn;

Unless I'm missing something, that certainly seems to explain
how a standby can lag behind even after a controlled shutdown of
the master.

best regards,
Florian Pflug



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] WIP: index support for regexp search

2013-04-15 Thread Alexander Korotkov
I found you committed GiST index implementation. That's cool.
I found an easy way to optimize it. We can also use trigramsMatchGraph for
signatures. Attached patch contains implementation.
Simple example in order to demonstrate it:

Before the patch:

test=# explain (analyze, buffers) select * from words where s ~ '[abc]def';
QUERY PLAN


--
 Bitmap Heap Scan on words  (cost=4.36..40.24 rows=10 width=9) (actual
time=17.189..17.193 rows=3 loops=1)
   Recheck Cond: (s ~ '[abc]def'::text)
   Buffers: shared hit=858
   -  Bitmap Index Scan on words_trgm_idx  (cost=0.00..4.36 rows=10
width=0) (actual time=17.172..17.172 rows=3 loops=1)
 Index Cond: (s ~ '[abc]def'::text)
 Buffers: shared hit=*857*
 Total runtime: 17.224 ms
(7 rows)

After the patch:

test=# explain (analyze, buffers) select * from words where s ~ '[abc]def';
QUERY PLAN


--
 Bitmap Heap Scan on words  (cost=4.36..40.24 rows=10 width=9) (actual
time=13.718..13.721 rows=3 loops=1)
   Recheck Cond: (s ~ '[abc]def'::text)
   Buffers: shared hit=498
   -  Bitmap Index Scan on words_trgm_idx  (cost=0.00..4.36 rows=10
width=0) (actual time=13.701..13.701 rows=3 loops=1)
 Index Cond: (s ~ '[abc]def'::text)
 Buffers: shared hit=*497*
 Total runtime: 13.786 ms
(7 rows)


--
With best regards,
Alexander Korotkov.


trgm-regexp-gist-optimize.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
COPY cannot be optimised correctly if we have before triggers or
volatile default expressions.

The multi-insert code detects those cases and falls back to the single
row mechanism in those cases.

There a common class of volatile functions that wouldn't cause
problems: any volatile function that doesn't touch the table being
loaded and still works correctly when called with alternately ordered
data.

I claim this is a common class, since sequence next_val functions and
uuid generators meet that criteria and most common forms of auditing
trigger, as well as any other form of data-reformatting trigger. Since
this is a common case, it seems worth optimising.

What I'd like to do is to invent a new form of labelling that allows
us to understand that COPY can still be optimised. I'm thinking to add
a new function label, something like one of
* IDEMPOTENT
* ORDER INDEPENDENT
* BATCHABLE
* NON SELF REFERENCING
* GO FASTER DAMMIT
etc

I'm sure many people will have a much more exact description and a
better name than I do.

This becomes more important when we think about parallelising SQL,
since essentially the same problem exists with parallel SQL calling
volatile functions. Oracle handles that by having a pragma to allow a
function to be declared as parallel safe.

I was also thinking that the best way to do this would be to invent a
new flexible function labelling scheme, so use something like hstore
to store a list of function attributes. Something that would mean we
don't have to invent new keywords every time we have a new function
label.

Suggestions please.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread hubert depesz lubaczewski
In current 9.3, I see:

$ select p.proname, p.provolatile from pg_proc p join pg_namespace n on 
p.pronamespace = n.oid where n.nspname = 'pg_catalog' and p.proname ~ 'json';
  proname  | provolatile 
---+-
 json_in   | s
 json_out  | i
 json_recv | s
 json_send | s
 array_to_json | s
 array_to_json | s
 row_to_json   | s
 row_to_json   | s
 json_agg_transfn  | i
 json_agg_finalfn  | i
 json_agg  | i
 to_json   | s
 json_object_field | s
 json_object_field_text| s
 json_array_element| s
 json_array_element_text   | s
 json_extract_path | s
 json_extract_path_op  | s
 json_extract_path_text| s
 json_extract_path_text_op | s
 json_array_elements   | s
 json_array_length | s
 json_object_keys  | s
 json_each | s
 json_each_text| s
 json_populate_record  | s
 json_populate_recordset   | s
(27 rows)

Is there any particular reason extract functions
(object_field/array_element/...) can't be immutable?

I can't readily imagine a situation where output of these functions would
change for different queries.

Best regards,

depesz

-- 
The best thing about modern society is how easy it is to avoid contact with it.
 http://depesz.com/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 03:00:34PM +0100, Simon Riggs wrote:
 COPY cannot be optimised correctly if we have before triggers or
 volatile default expressions.
 
 The multi-insert code detects those cases and falls back to the
 single row mechanism in those cases.
 
 There a common class of volatile functions that wouldn't cause
 problems: any volatile function that doesn't touch the table being
 loaded and still works correctly when called with alternately
 ordered data.

Doesn't touch already-existing rows?  Makes a lot of sense :)

 I claim this is a common class, since sequence next_val functions and
 uuid generators meet that criteria and most common forms of auditing
 trigger, as well as any other form of data-reformatting trigger. Since
 this is a common case, it seems worth optimising.

Do you have numbers on this, or ways to gather same?  In other words,
how do we know what resources (time, CPU cycles, disk seeks, etc.) are
being consumed here?

 What I'd like to do is to invent a new form of labelling that allows
 us to understand that COPY can still be optimised. I'm thinking to add
 a new function label, something like one of
 * IDEMPOTENT
 * ORDER INDEPENDENT
 * BATCHABLE
 * NON SELF REFERENCING
 * GO FASTER DAMMIT
 etc
 
 I'm sure many people will have a much more exact description and a
 better name than I do.
 
 This becomes more important when we think about parallelising SQL,
 since essentially the same problem exists with parallel SQL calling
 volatile functions. Oracle handles that by having a pragma to allow a
 function to be declared as parallel safe.

What happens when you misinform Oracle about this?  Does it attempt to
check?  More importantly, what *should* happen?

 I was also thinking that the best way to do this would be to invent a
 new flexible function labelling scheme, so use something like hstore
 to store a list of function attributes. Something that would mean we
 don't have to invent new keywords every time we have a new function
 label.
 
 Suggestions please.

JSON's in core.  How about using that?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] WIP: index support for regexp search

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 05:53:41PM +0400, Alexander Korotkov wrote:
 I found you committed GiST index implementation. That's cool.
 I found an easy way to optimize it. We can also use trigramsMatchGraph for
 signatures. Attached patch contains implementation.
 Simple example in order to demonstrate it:
 
 Before the patch:
  Buffers: shared hit=*857*
 After the patch:
  Buffers: shared hit=*497*

Neato!

Inside the patch, s/monotonous/monotone/, I think.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Andrew Dunstan


On 04/15/2013 11:16 AM, hubert depesz lubaczewski wrote:

In current 9.3, I see:

$ select p.proname, p.provolatile from pg_proc p join pg_namespace n on 
p.pronamespace = n.oid where n.nspname = 'pg_catalog' and p.proname ~ 'json';
   proname  | provolatile
---+-
  json_in   | s
  json_out  | i
  json_recv | s
  json_send | s
  array_to_json | s
  array_to_json | s
  row_to_json   | s
  row_to_json   | s
  json_agg_transfn  | i
  json_agg_finalfn  | i
  json_agg  | i
  to_json   | s
  json_object_field | s
  json_object_field_text| s
  json_array_element| s
  json_array_element_text   | s
  json_extract_path | s
  json_extract_path_op  | s
  json_extract_path_text| s
  json_extract_path_text_op | s
  json_array_elements   | s
  json_array_length | s
  json_object_keys  | s
  json_each | s
  json_each_text| s
  json_populate_record  | s
  json_populate_recordset   | s
(27 rows)

Is there any particular reason extract functions
(object_field/array_element/...) can't be immutable?

I can't readily imagine a situation where output of these functions would
change for different queries.



Me either. It's an oversight, really. Unless there is any objection I'll 
change them toot sweet. What about the existing (as of 9.2) functions?


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread hubert depesz lubaczewski
On Mon, Apr 15, 2013 at 11:31:39AM -0400, Andrew Dunstan wrote:
 Me either. It's an oversight, really. Unless there is any objection
 I'll change them toot sweet. What about the existing (as of 9.2)
 functions?

I don't think that 9.2 functions are that interesting, since these are
to build json values, and as such are not really candidates to making
index off.

Best regards,

depesz



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Heikki Linnakangas

On 15.04.2013 17:00, Simon Riggs wrote:

COPY cannot be optimised correctly if we have before triggers or
volatile default expressions.

The multi-insert code detects those cases and falls back to the single
row mechanism in those cases.

There a common class of volatile functions that wouldn't cause
problems: any volatile function that doesn't touch the table being
loaded and still works correctly when called with alternately ordered
data.

I claim this is a common class, since sequence next_val functions and
uuid generators meet that criteria and most common forms of auditing
trigger, as well as any other form of data-reformatting trigger. Since
this is a common case, it seems worth optimising.

What I'd like to do is to invent a new form of labelling that allows
us to understand that COPY can still be optimised.


It would be even nicer to detect at runtime, when a default expression 
or before trigger tries to access the same table. When that happens, we 
could immediately flush all the tuples buffered that far to disk, so 
that they are visible to the expression, and then proceed with it.


- Heikki


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes:
 Is there any particular reason extract functions
 (object_field/array_element/...) can't be immutable?
 
 I can't readily imagine a situation where output of these functions would
 change for different queries.

 Me either. It's an oversight, really. Unless there is any objection I'll 
 change them toot sweet. What about the existing (as of 9.2) functions?

Note that anything that invokes other types' I/O functions can't be
immutable, at most stable.  But the ones that don't do that certainly
should be fixed.

BTW, while I'm looking at this: json_populate_record and
json_populate_recordset are marked as non-strict, but they are
completely broken for null inputs because they attempt to fetch inputs
before checking them for null-ness.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Andres Freund
On 2013-04-15 11:31:39 -0400, Andrew Dunstan wrote:
 
 On 04/15/2013 11:16 AM, hubert depesz lubaczewski wrote:
 In current 9.3, I see:
 
 $ select p.proname, p.provolatile from pg_proc p join pg_namespace n on 
 p.pronamespace = n.oid where n.nspname = 'pg_catalog' and p.proname ~ 'json';
proname  | provolatile
 ---+-
   json_in   | s
   json_out  | i
   json_recv | s
   json_send | s
   array_to_json | s
   array_to_json | s
   row_to_json   | s
   row_to_json   | s
   json_agg_transfn  | i
   json_agg_finalfn  | i
   json_agg  | i
   to_json   | s
   json_object_field | s
   json_object_field_text| s
   json_array_element| s
   json_array_element_text   | s
   json_extract_path | s
   json_extract_path_op  | s
   json_extract_path_text| s
   json_extract_path_text_op | s
   json_array_elements   | s
   json_array_length | s
   json_object_keys  | s
   json_each | s
   json_each_text| s
   json_populate_record  | s
   json_populate_recordset   | s
 (27 rows)
 
 Is there any particular reason extract functions
 (object_field/array_element/...) can't be immutable?
 
 I can't readily imagine a situation where output of these functions would
 change for different queries.
 
 
 Me either. It's an oversight, really. Unless there is any objection I'll
 change them toot sweet. What about the existing (as of 9.2) functions?

ISTM json_in, out, recv, send should also be immutable. array_to_json,
row_to_json et all can't be tho.

Greetings,

Andres Freund

-- 
 Andres Freund http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:24, David Fetter da...@fetter.org wrote:

 I claim this is a common class, since sequence next_val functions and
 uuid generators meet that criteria and most common forms of auditing
 trigger, as well as any other form of data-reformatting trigger. Since
 this is a common case, it seems worth optimising.

 Do you have numbers on this, or ways to gather same?  In other words,
 how do we know what resources (time, CPU cycles, disk seeks, etc.) are
 being consumed here?

The multi-insert optimisation for COPY is already there and works well
enough to have been committed.

All we have to do to allow it to be used is to persuade COPY that come
kinds of volatile function need not prevent the optimisation. So once
we have a mechanism for appropriately labelling a function, it will be
a one-line change in copy.c to enable the optimisation.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes:
 COPY cannot be optimised correctly if we have before triggers or
 volatile default expressions.

 The multi-insert code detects those cases and falls back to the single
 row mechanism in those cases.

 There a common class of volatile functions that wouldn't cause
 problems: any volatile function that doesn't touch the table being
 loaded and still works correctly when called with alternately ordered
 data.

 I claim this is a common class, since sequence next_val functions and
 uuid generators meet that criteria and most common forms of auditing
 trigger, as well as any other form of data-reformatting trigger.

I don't believe that it's a good idea to consider nextval() to be
reorderable, so I'm not convinced by your argument here.

 What I'd like to do is to invent a new form of labelling that allows
 us to understand that COPY can still be optimised.

And I don't want to invent impossible-to-verify function attributes with
such a tiny use-case as this.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:41, Heikki Linnakangas hlinnakan...@vmware.com wrote:

 What I'd like to do is to invent a new form of labelling that allows
 us to understand that COPY can still be optimised.

 It would be even nicer to detect at runtime, when a default expression or
 before trigger tries to access the same table. When that happens, we could
 immediately flush all the tuples buffered that far to disk, so that they are
 visible to the expression, and then proceed with it.

Maybe, but do we really want an extra test every time we access a
table? And if we did that, how would we pass control back to the COPY
command to allow it flush the buffer before continuing with the
function?

How would we cope with times when a subtle problem makes a function
unusable, even though it passes automatic detection?

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes:
 On 15 April 2013 16:24, David Fetter da...@fetter.org wrote:
 Do you have numbers on this, or ways to gather same?  In other words,
 how do we know what resources (time, CPU cycles, disk seeks, etc.) are
 being consumed here?

 The multi-insert optimisation for COPY is already there and works well
 enough to have been committed.

You seem to not have answered the question.  Exactly what sort of
performance gain might be possible, bearing in mind that anything that
invokes a trigger (for instance) is unlikely to be amazingly fast
anyway?

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote:
 Simon Riggs si...@2ndquadrant.com writes:
 On 15 April 2013 16:24, David Fetter da...@fetter.org wrote:
 Do you have numbers on this, or ways to gather same?  In other words,
 how do we know what resources (time, CPU cycles, disk seeks, etc.) are
 being consumed here?

 The multi-insert optimisation for COPY is already there and works well
 enough to have been committed.

 You seem to not have answered the question.  Exactly what sort of
 performance gain might be possible, bearing in mind that anything that
 invokes a trigger (for instance) is unlikely to be amazingly fast
 anyway?

Forgive me, I assumed the list would be familiar with the optimization
and so be excited by the need for this.

I will implement as a kluge, test and report the results.

Loading data into a table with a SERIAL or UUID column is the main use
case, so I'll measure that.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 11:49:42AM -0400, Tom Lane wrote:
 Simon Riggs si...@2ndquadrant.com writes:
  COPY cannot be optimised correctly if we have before triggers or
  volatile default expressions.
 
  The multi-insert code detects those cases and falls back to the single
  row mechanism in those cases.
 
  There a common class of volatile functions that wouldn't cause
  problems: any volatile function that doesn't touch the table being
  loaded and still works correctly when called with alternately ordered
  data.
 
  I claim this is a common class, since sequence next_val functions and
  uuid generators meet that criteria and most common forms of auditing
  trigger, as well as any other form of data-reformatting trigger.
 
 I don't believe that it's a good idea to consider nextval() to be
 reorderable, so I'm not convinced by your argument here.

We tell people very clearly in the docs and elsewhere that nextval()
guarantees uniqueness and very specifically not ordering, so with
greatest respect, I agree with Simon on its reorderability.

  What I'd like to do is to invent a new form of labelling that
  allows us to understand that COPY can still be optimised.
 
 And I don't want to invent impossible-to-verify function attributes
 with such a tiny use-case as this.

Are you referring to the Halting Problem?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote:
 On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote:
  Simon Riggs si...@2ndquadrant.com writes:
  On 15 April 2013 16:24, David Fetter da...@fetter.org wrote:
  Do you have numbers on this, or ways to gather same?  In other
  words, how do we know what resources (time, CPU cycles, disk
  seeks, etc.) are being consumed here?
 
  The multi-insert optimisation for COPY is already there and works
  well enough to have been committed.
 
  You seem to not have answered the question.  Exactly what sort of
  performance gain might be possible, bearing in mind that anything
  that invokes a trigger (for instance) is unlikely to be amazingly
  fast anyway?
 
 Forgive me, I assumed the list would be familiar with the
 optimization and so be excited by the need for this.
 
 I will implement as a kluge, test and report the results.
 
 Loading data into a table with a SERIAL or UUID column is the main
 use case, so I'll measure that.

The former is common enough a use case to optimize specifically,
should the numbers come out right.  Do you suppose that an in-core
UUID generator would help the latter make more sense as a part of the
same use case?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:08, David Fetter da...@fetter.org wrote:

 Loading data into a table with a SERIAL or UUID column is the main
 use case, so I'll measure that.

 The former is common enough a use case to optimize specifically,
 should the numbers come out right.  Do you suppose that an in-core
 UUID generator would help the latter make more sense as a part of the
 same use case?

Only if some form of labelling becomes an issue, but I'm ever hopeful.
Let's wait for the test results now.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Hannu Krosing

On 04/15/2013 06:04 PM, Simon Riggs wrote:

On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote:

Simon Riggs si...@2ndquadrant.com writes:

On 15 April 2013 16:24, David Fetter da...@fetter.org wrote:

Do you have numbers on this, or ways to gather same?  In other words,
how do we know what resources (time, CPU cycles, disk seeks, etc.) are
being consumed here?

The multi-insert optimisation for COPY is already there and works well
enough to have been committed.

You seem to not have answered the question.  Exactly what sort of
performance gain might be possible, bearing in mind that anything that
invokes a trigger (for instance) is unlikely to be amazingly fast
anyway?

Forgive me, I assumed the list would be familiar with the optimization
and so be excited by the need for this.

I will implement as a kluge, test and report the results.

Would just declaring nextval() to be a stable function be a good test ?

Hannu


Loading data into a table with a SERIAL or UUID column is the main use
case, so I'll measure that.

--
  Simon Riggs   http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training  Services






--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
David Fetter da...@fetter.org writes:
 On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote:
 Loading data into a table with a SERIAL or UUID column is the main
 use case, so I'll measure that.

 The former is common enough a use case to optimize specifically,
 should the numbers come out right.

Yeah.  TBH I would rather see a special-case hack in the COPY code to
accept nextval() than expose anything as dirty and special-purpose as
this proposed flag to users.  But in any case, I don't believe that
adequate evidence has been offered to show that we should do anything
at all here.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] WIP: index support for regexp search

2013-04-15 Thread Tom Lane
Alexander Korotkov aekorot...@gmail.com writes:
 I found you committed GiST index implementation. That's cool.
 I found an easy way to optimize it. We can also use trigramsMatchGraph for
 signatures. Attached patch contains implementation.

Good idea, committed.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:04, Simon Riggs si...@2ndquadrant.com wrote:

 I will implement as a kluge, test and report the results.

Test is COPY 1 million rows on a table with 2 columns, both bigint.
Verified no checkpoints triggered during load.
No other work active on database, tests condicted on laptop
Autovacuum disabled.
Results from multiple runs, outliers excluded, rough averages

HEAD
COPY, with sequence ~5500ms
COPY, with sequence, cached ~5000ms
COPY, no sequence ~1600ms

PATCH to allow sequences to use multi-insert optimisation (1 line change)
COPY, with sequence ~1850ms
COPY, with sequence, cached ~1750ms
COPY, no sequence ~1600ms

This shows that
* cacheing the sequence gives a useful improvement currently
* use of multi-insert optimisaton is very important

Proposals
* set CACHE 100 on automatically created SERIAL sequences
* allow some way to use multi-insert optimisation when default expr is
next_val on a sequence

Tests performed without indexes since this is another area of known
performance issues that I hope to cover later. Zero indexes is not
real, but we're trying to measure the effect and benefit of an
isolated change, so in this case it is appropriate.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 06:30:55PM +0100, Simon Riggs wrote:
 On 15 April 2013 17:04, Simon Riggs si...@2ndquadrant.com wrote:
 
  I will implement as a kluge, test and report the results.
 
 Test is COPY 1 million rows on a table with 2 columns, both bigint.
 Verified no checkpoints triggered during load.
 No other work active on database, tests condicted on laptop
 Autovacuum disabled.
 Results from multiple runs, outliers excluded, rough averages
 
 HEAD
 COPY, with sequence ~5500ms
 COPY, with sequence, cached ~5000ms
 COPY, no sequence ~1600ms
 
 PATCH to allow sequences to use multi-insert optimisation (1 line change)
 COPY, with sequence ~1850ms
 COPY, with sequence, cached ~1750ms
 COPY, no sequence ~1600ms
 
 This shows that
 * cacheing the sequence gives a useful improvement currently
 * use of multi-insert optimisaton is very important
 
 Proposals
 * set CACHE 100 on automatically created SERIAL sequences
 * allow some way to use multi-insert optimisation when default expr is
 next_val on a sequence
 
 Tests performed without indexes since this is another area of known
 performance issues that I hope to cover later. Zero indexes is not
 real, but we're trying to measure the effect and benefit of an
 isolated change, so in this case it is appropriate.

The difference between HEAD and patch in the COPY, with sequence
case is pretty remarkable.  What's the patch?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 18:41, David Fetter da...@fetter.org wrote:

 The difference between HEAD and patch in the COPY, with sequence
 case is pretty remarkable.  What's the patch?

Attached.

This is usable only for this test. It is not anywhere remotely close
to being applied.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


copy_test_kluge.v1.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 07:04:55PM +0100, Simon Riggs wrote:
 On 15 April 2013 18:41, David Fetter da...@fetter.org wrote:
 
  The difference between HEAD and patch in the COPY, with sequence
  case is pretty remarkable.  What's the patch?
 
 Attached.

Thanks! :)

 This is usable only for this test. It is not anywhere remotely close
 to being applied.

Of course.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 I claim this is a common class, since sequence next_val functions and
 uuid generators meet that criteria and most common forms of auditing
 trigger, as well as any other form of data-reformatting trigger.

 I don't believe that it's a good idea to consider nextval() to be
 reorderable, so I'm not convinced by your argument here.

Why not?

I admit that I can't convince myself that it's safe.  But I can't
think of a concrete example where it breaks anything, either.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Query not using index for user defined type

2013-04-15 Thread Rodrigo Barboza
Hi guys.
I created a type 'mytype' (an unsigned int) and created an operator class
for index.
Then I created a table with a column of my type and isnerted 1000 entries.
But no matter how many entries I have in the table, it never uses the
index. It always does a seq scan.

Here is the explain analyze with 1000 entries:

explain analyze select * from mytable where a  120::mytype and a 
530::mytype;

---
 Seq Scan on mytable  (cost=0.00..19.02 rows=400 width=4) (actual
time=0.023..0.229 rows=409 loops=1)
   Filter: ((a  '120'::mytype) AND (a  '530'::mytpe))
 Total runtime: 0.297 ms


Does anybody know why?


Re: [HACKERS] [PATCH] pg_regress and non-default unix socket path

2013-04-15 Thread Robert Haas
On Fri, Apr 12, 2013 at 1:00 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 The hunk that changes the messages might need some thought so that it
 doesn't cause a translation regression.  But in general I see no
 reason not to do this before we release beta1.  It seems safe enough,
 and changes that reduce the need for packagers to carry private
 patches are, I think, generally a good thing.

 It looks to me like this is asking for pg_regress to adopt a nonstandard
 interpretation of PGHOST, which doesn't seem like a wise thing at all,
 especially if it's not documented.

I see it the other way around.  Most places in PostgreSQL that allow a
hostname also allow a string beginning with a slash to be specified
instead, which then gets interpreted as a socket directory name.
pg_regress does not allow that, and this patch would fix that.

 FWIW, the equivalent thing in the Red Hat/Fedora packages can be seen
 in this patch:

 http://pkgs.fedoraproject.org/cgit/postgresql.git/plain/postgresql-var-run-socket.patch

 which would not get noticeably shorter if we hacked pg_regress in the
 suggested way.  AFAICT, instead of touching pg_regress.c, Red Hat's
 patch would need to do something to the regression Makefiles if we
 wanted to use this implementation.  I'm not convinced that'd be better
 at all.  TBH, if this is committed, the Red Hat patches will probably
 end up reverting it.

The Red Hat patch is aiming to change the run-time behavior of the
server, which Christoph's patch is not.  The net effect would be that
the last two hunks could be ditched in favor of setting
EXTRA_REGRESS_OPTS.  I don't imagine that's a big improvement but it
doesn't seem like a step backward, either.  I can certainly see the
appeal: IME, it's much nicer to pass in a few extra configuration
options than to have to patch the source.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 I don't believe that it's a good idea to consider nextval() to be
 reorderable, so I'm not convinced by your argument here.

 Why not?

 I admit that I can't convince myself that it's safe.  But I can't
 think of a concrete example where it breaks anything, either.

I think plenty of people would be upset if row serial numbers assigned
with nextval() were not assigned in the order of the incoming rows.
The argument that you can get gaps in the sequence in some corner cases
(none of which apply within a single COPY operation, btw) doesn't
entitle us to violate the POLA to that extent.

After looking again at the code Simon is concerned about, though,
whether we are willing to allow volatile function calls to be reordered
has approximately nothing to do with this COPY optimization.  Rather,
the thing that makes it safe is that nextval() isn't going to look at
the COPY target table, and thus whether or not the previous rows have
been physically inserted isn't important.  The same goes for the UUID
example.  So I think he's done himself a disservice by talking about
reordering and bringing up the question of parallel queries.  What we
ought to be thinking about is how we can be certain that a function call
isn't going to look at the uncommitted table rows; no more and no less.

In this context, I think we could do a lot worse than to special-case
nextval(), because it's hard to see a really principled function
attribute definition that would admit it here.  It does look at, and
even modify, uncommitted database state.  We know it's safe because a
sequence relation couldn't be the target of COPY ... but that reasoning
doesn't fit nicely into anything I think we'd want to expose to users.

OTOH, the notion that a UUID generator doesn't touch *any* database
state seems like it might be worth treating as a general function
property: it's simple to understand and applies to a lot of other
volatile functions such as random() and clock_timestamp().

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Jaime Casanova
On Mon, Apr 15, 2013 at 3:08 PM, Rodrigo Barboza
rodrigombu...@gmail.com wrote:

 Here is the explain analyze with 1000 entries:

 explain analyze select * from mytable where a  120::mytype and a 
 530::mytype;


I'm not sure this is appropiate for -hackers, maybe should post on -general.
Also provide scripts with the creation of the datatype and/or the
OPERATOR FAMILY

anyway, this explain is completely useless to prove your point. if you
are selecting almost half of the table it will use a seq scan. try
again with more rows in the table and a narrow filter

--
Jaime Casanova www.2ndQuadrant.com
Professional PostgreSQL: Soporte 24x7 y capacitaciĆ³n
Phone: +593 4 5107566 Cell: +593 987171157


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Jaime Casanova
On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote:

 OTOH, the notion that a UUID generator doesn't touch *any* database
 state seems like it might be worth treating as a general function
 property: it's simple to understand and applies to a lot of other
 volatile functions such as random() and clock_timestamp().


Something like the NO SQL access indication mandated by sql standard?

http://www.postgresql.org/message-id/1267473390.7837.9.ca...@vanquo.pezone.net

--
Jaime Casanova www.2ndQuadrant.com
Professional PostgreSQL: Soporte 24x7 y capacitaciĆ³n
Phone: +593 4 5107566 Cell: +593 987171157


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Tom Lane
Rodrigo Barboza rodrigombu...@gmail.com writes:
 I created a type 'mytype' (an unsigned int) and created an operator class
 for index.
 Then I created a table with a column of my type and isnerted 1000 entries.
 But no matter how many entries I have in the table, it never uses the
 index. It always does a seq scan.

 Here is the explain analyze with 1000 entries:

 explain analyze select * from mytable where a  120::mytype and a 
 530::mytype;

 ---
  Seq Scan on mytable  (cost=0.00..19.02 rows=400 width=4) (actual
 time=0.023..0.229 rows=409 loops=1)
Filter: ((a  '120'::mytype) AND (a  '530'::mytpe))
  Total runtime: 0.297 ms

Using a seqscan to fetch 400 out of 1000 entries is the right thing.
(The crossover point where an index becomes unhelpful is a lot closer
to 1% of the table than it is to 40%.)

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 20:52, Robert Haas robertmh...@gmail.com wrote:
 On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 I claim this is a common class, since sequence next_val functions and
 uuid generators meet that criteria and most common forms of auditing
 trigger, as well as any other form of data-reformatting trigger.

 I don't believe that it's a good idea to consider nextval() to be
 reorderable, so I'm not convinced by your argument here.

 Why not?

 I admit that I can't convince myself that it's safe.  But I can't
 think of a concrete example where it breaks anything, either.


I'm not sure exactly if the corect label is reorderable or some
other word that describes this specific situation. Parallel
independent probably would work, since if we had a DML statement that
worked in 2 separate processes we'd probably want to know that the
rows in each could be divided up without changing the result.

It looks straightforward to put in a special case check for sequences,
which is the most important use case. Sequences are a recognised
database object, so such a special case could be justified... shame
about the other use cases though, e.g. UUIDs.

I guess we can generalise it when we have a better idea of whether
this does indeed make a useful generalisation.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 4:21 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 I think plenty of people would be upset if row serial numbers assigned
 with nextval() were not assigned in the order of the incoming rows.
 The argument that you can get gaps in the sequence in some corner cases
 (none of which apply within a single COPY operation, btw) doesn't
 entitle us to violate the POLA to that extent.

I agree.

 After looking again at the code Simon is concerned about, though,
 whether we are willing to allow volatile function calls to be reordered
 has approximately nothing to do with this COPY optimization.  Rather,
 the thing that makes it safe is that nextval() isn't going to look at
 the COPY target table, and thus whether or not the previous rows have
 been physically inserted isn't important.  The same goes for the UUID
 example.  So I think he's done himself a disservice by talking about
 reordering and bringing up the question of parallel queries.  What we
 ought to be thinking about is how we can be certain that a function call
 isn't going to look at the uncommitted table rows; no more and no less.

Yep.

 In this context, I think we could do a lot worse than to special-case
 nextval(), because it's hard to see a really principled function
 attribute definition that would admit it here.  It does look at, and
 even modify, uncommitted database state.  We know it's safe because a
 sequence relation couldn't be the target of COPY ... but that reasoning
 doesn't fit nicely into anything I think we'd want to expose to users.

 OTOH, the notion that a UUID generator doesn't touch *any* database
 state seems like it might be worth treating as a general function
 property: it's simple to understand and applies to a lot of other
 volatile functions such as random() and clock_timestamp().

I think that's right; and I also think that's something that could be
useful in other contexts.  It also has the advantage of being a
*checkable* property.  That is, if a function is marked as changing no
database state, we can set a flag on entry into that context and clear
the flag on exit.  If in the middle any attempt is made to change the
database state, then we can throw an error.  This would be pretty good
insurance against the possibility that future optimizations based
around that flag would cause behavioral differences.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Andrew Dunstan


On 04/15/2013 11:46 AM, Andres Freund wrote:


Me either. It's an oversight, really. Unless there is any objection I'll
change them toot sweet. What about the existing (as of 9.2) functions?

ISTM json_in, out, recv, send should also be immutable. array_to_json,
row_to_json et all can't be tho.



OK, although these have been like this since 9.2. I'm not sure why 
json_out is immutable but json_in isn't.


Does changing these require a catalog version bump?

cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Rodrigo Barboza
On Mon, Apr 15, 2013 at 5:33 PM, Tom Lane t...@sss.pgh.pa.us wrote:

 Rodrigo Barboza rodrigombu...@gmail.com writes:
  I created a type 'mytype' (an unsigned int) and created an operator class
  for index.
  Then I created a table with a column of my type and isnerted 1000
 entries.
  But no matter how many entries I have in the table, it never uses the
  index. It always does a seq scan.

  Here is the explain analyze with 1000 entries:

  explain analyze select * from mytable where a  120::mytype and a 
  530::mytype;

 
 ---
   Seq Scan on mytable  (cost=0.00..19.02 rows=400 width=4) (actual
  time=0.023..0.229 rows=409 loops=1)
 Filter: ((a  '120'::mytype) AND (a  '530'::mytpe))
   Total runtime: 0.297 ms

 Using a seqscan to fetch 400 out of 1000 entries is the right thing.
 (The crossover point where an index becomes unhelpful is a lot closer
 to 1% of the table than it is to 40%.)

 regards, tom lane



You were right Tom, when I did  200 it used the index.

But I have another question.
I created a implic cast for mytype to bigint.
So when I do the same query it does seq scan, because the column is
transformed into bigint.
Is there a good solution for this?
Here is an examples with the 2 queries.

explain analyze select * from mytable where a  200::mytype;
 QUERY PLAN

 Index Scan using mytype_idx on tm32  (cost=0.00..11.66 rows=195 width=4)
(actual time=0.020..0.068 rows=200 loops=1)
   Index Cond: (a  '200'::mytype)
 Total runtime: 0.111 ms


explain analyze select * from mytable where a  200;
QUERY PLAN
---
 Seq Scan on mytable  (cost=0.00..19.02 rows=334 width=4) (actual
time=0.015..0.215 rows=200 loops=1)
   Filter: ((a)::bigint  200)
 Total runtime: 0.238 ms


Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 21:32, Jaime Casanova ja...@2ndquadrant.com wrote:
 On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote:

 OTOH, the notion that a UUID generator doesn't touch *any* database
 state seems like it might be worth treating as a general function
 property: it's simple to understand and applies to a lot of other
 volatile functions such as random() and clock_timestamp().


 Something like the NO SQL access indication mandated by sql standard?

 http://www.postgresql.org/message-id/1267473390.7837.9.ca...@vanquo.pezone.net

That would work for UUIDs, random() etc but not for sequences.

So I'll treat this as two separate cases:
* add special case for sequences
* use the NO SQL mechanism, as described, which implies no reads or
writes of database state. We could test that, but its somewhat harder
and we'd need to test for that on entry to any function, which I don't
much like.

Default to current timestamp is also a common use case - thanks for
mentioning that.

Doing it tha way Tatsuo would be able to parse functions more easily
as requested in the linked post.

--
 Simon Riggs   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread anara...@anarazel.de


Andrew Dunstan and...@dunslane.net schrieb:


On 04/15/2013 11:46 AM, Andres Freund wrote:

 Me either. It's an oversight, really. Unless there is any objection
I'll
 change them toot sweet. What about the existing (as of 9.2)
functions?
 ISTM json_in, out, recv, send should also be immutable.
array_to_json,
 row_to_json et all can't be tho.


OK, although these have been like this since 9.2. I'm not sure why 
json_out is immutable but json_in isn't.

Does changing these require a catalog version bump?

Well, you could get away without one since a more permissive value should only 
influence performance and not correctness. But there doesn't yet seem much 
reason to avoid it that much yet. It could cause confusion for someone at some 
point.

Andres

--- 
Please excuse brevity and formatting - I am writing this on my mobile phone.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Tom Lane
Rodrigo Barboza rodrigombu...@gmail.com writes:
 I created a implic cast for mytype to bigint.
 So when I do the same query it does seq scan, because the column is
 transformed into bigint.

Yeah.  One reason why there's not an unsigned int type already is that
it seems impossible to shoehorn it into the numeric promotion hierarchy
without breaking a lot of existing cases.  You definitely aren't likely
to get nice results by just adding some implicit casts without doing a
very careful design beforehand.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Query not using index for user defined type

2013-04-15 Thread Rodrigo Barboza
On Mon, Apr 15, 2013 at 7:51 PM, Tom Lane t...@sss.pgh.pa.us wrote:

 Rodrigo Barboza rodrigombu...@gmail.com writes:
  I created a implic cast for mytype to bigint.
  So when I do the same query it does seq scan, because the column is
  transformed into bigint.

 Yeah.  One reason why there's not an unsigned int type already is that
 it seems impossible to shoehorn it into the numeric promotion hierarchy
 without breaking a lot of existing cases.  You definitely aren't likely
 to get nice results by just adding some implicit casts without doing a
 very careful design beforehand.

 regards, tom lane


I just added implicit cast from my type to int8, numeric and float.
No implicit cast for lower level types.
Isn't it safe?
The problem would be only about the index?


Re: [HACKERS] Why are JSON extraction functions STABLE and not IMMUTABLE?

2013-04-15 Thread Bruce Momjian
On Mon, Apr 15, 2013 at 04:41:53PM -0400, Andrew Dunstan wrote:
 
 On 04/15/2013 11:46 AM, Andres Freund wrote:
 
 Me either. It's an oversight, really. Unless there is any objection I'll
 change them toot sweet. What about the existing (as of 9.2) functions?
 ISTM json_in, out, recv, send should also be immutable. array_to_json,
 row_to_json et all can't be tho.
 
 
 OK, although these have been like this since 9.2. I'm not sure why
 json_out is immutable but json_in isn't.
 
 Does changing these require a catalog version bump?

Yes.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] event trigger API documentation?

2013-04-15 Thread Peter Eisentraut
I'm having trouble finding documentation about how to write event
triggers.  The chapter in the documentation

http://www.postgresql.org/docs/devel/static/event-triggers.html

says they can be written in C or supported PLs, but does not explain it
any further.  Is there any documentation for it?




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers