[HACKERS] Missing optimization when filters are applied after window functions

2012-05-16 Thread Volker Grabsch
Dear PostgreSQL hackers, [ please CC to me as I'm not subscribed to the list ] I think there is a missing optimization when a filter is applied after a window function, where the filtered field is also used for partitioning. Here a simplified example: Suppose we have a table that stores 100.000

[HACKERS] 9.2 Beta: intersection of daterange

2012-05-16 Thread Brar Piening
I'm currently doing some tests on range types: tests=# SELECT int8range(5,15) * int8range(10,20) AS intersection; intersection -- [10,15) (1 Zeile) tests=# tests=# SELECT '[2010-03-15,2010-05-22)'::daterange * '[2010-01-04,)'::daterange AS intersection; intersection

Re: [HACKERS] 9.2 Beta: intersection of daterange

2012-05-16 Thread Misa Simic
I think result is ok... 2010-01-04 is not inside first range... Sent from my Windows Phone From: Brar Piening Sent: 16/05/2012 09:53 To: pgsql-hackers Subject: [HACKERS] 9.2 Beta: intersection of daterange I'm currently doing some tests on range types: tests=# SELECT int8range(5,15) *

Re: [HACKERS] 9.2 Beta: intersection of daterange

2012-05-16 Thread Brar Piening
Misa Simic wrote: I think result is ok... 2010-01-04 is not inside first range... Staring at my query for five minutes obviously didn't prevent me from creating that noise. I meant to query SELECT '[2010-03-15,2010-05-22)'::daterange * '[2010-04-01,)'::daterange AS intersection; which

[HACKERS] Interrupting long external library calls

2012-05-16 Thread Mark Cave-Ayland
Hi all, One of the issues we've been looking at with PostGIS is how to interrupt long-running processing tasks in external libraries, particularly GEOS. After a few tests here, it seems that even the existing SIGALRM handler doesn't get called if statement_timeout is reached when in an

Re: [HACKERS] Interrupting long external library calls

2012-05-16 Thread Heikki Linnakangas
On 16.05.2012 13:25, Mark Cave-Ayland wrote: One of the issues we've been looking at with PostGIS is how to interrupt long-running processing tasks in external libraries, particularly GEOS. After a few tests here, it seems that even the existing SIGALRM handler doesn't get called if

Re: [HACKERS] Interrupting long external library calls

2012-05-16 Thread Mark Cave-Ayland
On 16/05/12 11:39, Heikki Linnakangas wrote: One of the issues we've been looking at with PostGIS is how to interrupt long-running processing tasks in external libraries, particularly GEOS. After a few tests here, it seems that even the existing SIGALRM handler doesn't get called if

Re: [HACKERS] Interrupting long external library calls

2012-05-16 Thread Heikki Linnakangas
On 16.05.2012 14:30, Mark Cave-Ayland wrote: On 16/05/12 11:39, Heikki Linnakangas wrote: However, if you're absolutely positively sure that the library function can tolerate that, you can set ImmediateInterruptOK = true before calling it. See e.g PGSemaphoreLock() on how that's done before

[HACKERS] Avoiding execution of some functions by query rewriting

2012-05-16 Thread Thomas Girault
Hello, I would like to allow the execution of a function (my_function) only if its argument (my_table.x) belongs to a predefined interval (e.g. [100,1000]). Let's take the following query example : (q) SELECT * FROM my_table WHERE my_function(mytable.x); I would like this query automatically

Re: [HACKERS] Avoiding execution of some functions by query rewriting

2012-05-16 Thread Tom Lane
Thomas Girault toma.gira...@gmail.com writes: Hello, I would like to allow the execution of a function (my_function) only if its argument (my_table.x) belongs to a predefined interval (e.g. [100,1000]). Let's take the following query example : (q) SELECT * FROM my_table WHERE

Re: [HACKERS] Interrupting long external library calls

2012-05-16 Thread Sandro Santilli
On Wed, May 16, 2012 at 02:46:17PM +0300, Heikki Linnakangas wrote: On 16.05.2012 14:30, Mark Cave-Ayland wrote: On 16/05/12 11:39, Heikki Linnakangas wrote: However, if you're absolutely positively sure that the library function can tolerate that, you can set ImmediateInterruptOK = true

Re: [HACKERS] Avoiding execution of some functions by query rewriting

2012-05-16 Thread Florian Pflug
On May16, 2012, at 14:30 , Thomas Girault wrote: I would like to allow the execution of a function (my_function) only if its argument (my_table.x) belongs to a predefined interval (e.g. [100,1000]). Let's take the following query example : (q) SELECT * FROM my_table WHERE

Re: [HACKERS] Missing optimization when filters are applied after window functions

2012-05-16 Thread Tom Lane
Volker Grabsch v...@notjusthosting.com writes: I propose the following general optimization: If all window functions are partitioned by the same first field (here: id), then any filter on that field should be executed before WindowAgg. I'm not sure if that rule is correct in detail, but in

[HACKERS] read() returns ERANGE in Mac OS X

2012-05-16 Thread Alvaro Herrera
Hi, We just came across a situation where a corrupted HFS+ filesystem appears to return ERANGE on a customer machine. Our first reaction was to turn zero_damaged_pages on to allow taking a pg_dump backup of the database, but surprisingly this does not work. A quick glance at the code shows the

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-16 Thread Tom Lane
Alvaro Herrera alvhe...@alvh.no-ip.org writes: We just came across a situation where a corrupted HFS+ filesystem appears to return ERANGE on a customer machine. Our first reaction was to turn zero_damaged_pages on to allow taking a pg_dump backup of the database, but surprisingly this does

Re: [HACKERS] Missing optimization when filters are applied after window functions

2012-05-16 Thread Hitoshi Harada
On Wed, May 16, 2012 at 12:50 AM, Volker Grabsch v...@notjusthosting.com wrote: I propose the following general optimization: If all window functions are partitioned by the same first field (here: id), then any filter on that field should be executed before WindowAgg. So a query like this: I

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Fujii Masao
On Wed, May 16, 2012 at 2:29 AM, Thom Brown t...@linux.com wrote: On 15 May 2012 13:15, Fujii Masao masao.fu...@gmail.com wrote: On Wed, May 16, 2012 at 1:36 AM, Thom Brown t...@linux.com wrote: However, this isn't true when I restart the standby.  I've been informed that this should work fine

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-16 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mié may 16 09:51:26 -0400 2012: Alvaro Herrera alvhe...@alvh.no-ip.org writes: We just came across a situation where a corrupted HFS+ filesystem appears to return ERANGE on a customer machine. Our first reaction was to turn zero_damaged_pages on to

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Fujii Masao
On Wed, May 16, 2012 at 3:42 AM, Joshua Berkus j...@agliodbs.com wrote: Fujii, Wait, are you telling me that we *still* can't remaster from streaming replication? What's the remaster? And: if we still have to ship logs, what's the point in even having cascading replication? At least

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Fujii Masao
On Wed, May 16, 2012 at 3:43 AM, Joshua Berkus j...@agliodbs.com wrote: Before restarting it, you need to do pg_basebackup and make a base backup onto the standby again. Since you started the standby without recovery.conf, a series of WAL in the standby has gotten inconsistent with that in

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Thom Brown
On 16 May 2012 11:36, Fujii Masao masao.fu...@gmail.com wrote: On Wed, May 16, 2012 at 2:29 AM, Thom Brown t...@linux.com wrote: On 15 May 2012 13:15, Fujii Masao masao.fu...@gmail.com wrote: On Wed, May 16, 2012 at 1:36 AM, Thom Brown t...@linux.com wrote: However, this isn't true when I

Re: [HACKERS] Pre-alloc ListCell's optimization

2012-05-16 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: ISTM the first thing we'd need to have before we could think about this rationally is some measurements about the frequencies of different List lengths in a typical workload. I agree, that'd be a good

Re: [HACKERS] Interrupting long external library calls

2012-05-16 Thread Heikki Linnakangas
On 16.05.2012 15:42, Sandro Santilli wrote: But CHECK_FOR_INTERRUPTS doesn't return, right ? Is there another macro for just checking w/out yet acting upon it ? Hmm, no. CHECK_FOR_INTERRUPTS() checks the InterruptPending variable, but on Windows it also checks for UNBLOCKED_SIGNAL_QUEUE().

Re: [HACKERS] Pre-alloc ListCell's optimization

2012-05-16 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: So, when it comes to palloc() reduction, this patch would eliminate 99% of palloc's due to lists. For the regression tests, we're talking about reducing 893,206 palloc calls to only 1. Apologies, that wasn't quite right- it'd reduce it to 1 palloc

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-05-16 Thread Kyotaro HORIGUCHI
Hello, On Sun, May 13, 2012 at 10:38 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I've overlooked that startup process of the standby reads archives first, and then WAL. But the current patch enables progress governing based on checkpoint_segments during archive recovery on

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Jim Nasby
Well, that is a form of testing. :) My point was that we need some kind of regression tests around all the new replication stuff, and if you had some scripts that would be a useful starting point. But it sounds like you haven't gotten that far with it, so... On 5/15/12 10:12 AM, Joshua Berkus

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Jim Nasby
On 5/16/12 10:53 AM, Fujii Masao wrote: On Wed, May 16, 2012 at 3:43 AM, Joshua Berkusj...@agliodbs.com wrote: Before restarting it, you need to do pg_basebackup and make a base backup onto the standby again. Since you started the standby without recovery.conf, a series of WAL in the standby

Re: [HACKERS] Pre-alloc ListCell's optimization

2012-05-16 Thread Stephen Frost
Tom, * Stephen Frost (sfr...@snowman.net) wrote: Second, there are a couple of bugs (or at least, I'll characterize them that way) where we're pfree'ing a list which has been passed to list_concat(). That's not strictly legal as either argument passed to list_concat() could be

[HACKERS] psql bug

2012-05-16 Thread Teodor Sigaev
After editing query with external editor psql exits on Ctrl-C: % psql postgres SET Timing is on. psql (9.2beta1) Type help for help. postgres=# --- Ctrl-C postgres=# --- Ctrl-C postgres=# --- Ctrl-C postgres=# \e ... postgres=# --- Ctrl-C % Some details: external editor - nvi/vim, OS

Re: [HACKERS] Pre-alloc ListCell's optimization

2012-05-16 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: The two cases in clauses.c are pretty specific and documented: List *subargs = list_copy(((BoolExpr *) arg)-args); /* overly tense code to avoid leaking unused list header */ if (!unprocessed_args) unprocessed_args =

Re: [HACKERS] Draft release notes complete

2012-05-16 Thread Jeff Janes
On Wed, May 9, 2012 at 8:11 PM, Bruce Momjian br...@momjian.us wrote: I have completed my draft of the 9.2 release notes, and committed it to git.  I am waiting for our development docs to build, but after 40 minutes, I am still waiting:        

Re: [HACKERS] Draft release notes complete

2012-05-16 Thread Heikki Linnakangas
On 16.05.2012 22:38, Jeff Janes wrote: For item: Improve COPY performance by adding tuples to the heap in batches (Heikki Linnakangas) I think we should point out that the batching only applies for COPY into unindexed tables. Nice as the feature is, that is pretty big limitation not to

Re: [HACKERS] Pre-alloc ListCell's optimization

2012-05-16 Thread Stephen Frost
All, Here's an updated version of the patch which cleans up a couple of the previous issues, including addressing some of the free'ing issues. Looking forward to comments. Thanks, Stephen diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c

[HACKERS] temporal support patch

2012-05-16 Thread Miroslav Šimulčík
Hi all, as a part of my master's thesis I have created temporal support patch for PostgreSQL. It enables the creation of special temporal tables with entries versioning. Modifying operations (UPDATE, DELETE, TRUNCATE) on these tables don't cause permanent changes to entries, but create new

Re: [HACKERS] Draft release notes complete

2012-05-16 Thread Bruce Momjian
I will make the adjustments outlined below as soon as I can. --- On Sun, May 13, 2012 at 12:37:52AM -0400, Robert Haas wrote: On Sat, May 12, 2012 at 8:11 PM, Euler Taveira eu...@timbira.com wrote: On 12-05-2012 10:27,

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Fujii Masao
On Thu, May 17, 2012 at 1:07 AM, Thom Brown t...@linux.com wrote: On 16 May 2012 11:36, Fujii Masao masao.fu...@gmail.com wrote: On Wed, May 16, 2012 at 2:29 AM, Thom Brown t...@linux.com wrote: On 15 May 2012 13:15, Fujii Masao masao.fu...@gmail.com wrote: On Wed, May 16, 2012 at 1:36 AM,

Re: [HACKERS] psql bug

2012-05-16 Thread Tom Lane
Teodor Sigaev teo...@sigaev.ru writes: After editing query with external editor psql exits on Ctrl-C: FWIW, I failed to reproduce that on any of my machines. Maybe your editor is leaving the tty in a funny state? regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] transformations between types and languages

2012-05-16 Thread Robert Haas
On Tue, May 15, 2012 at 4:15 PM, Peter Eisentraut pete...@gmx.net wrote: [ draft design for the transforms feature ] Seems pretty reasonable, although I'm not sure about your chosen syntax for CREATE TRANSFORM... -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: [HACKERS] Strange issues with 9.2 pg_basebackup replication

2012-05-16 Thread Joshua Berkus
And: if we still have to ship logs, what's the point in even having cascading replication? At least cascading replication (1) allows you to adopt more flexible configuration of servers, I'm just pretty shocked. The last time we talked about this, at the end of the 9.1 development

[HACKERS] Why is indexonlyscan so darned slow?

2012-05-16 Thread Joshua Berkus
So, I set up a test which should have been ideal setup for index-only scan. The index was 1/10 the size of the table, and fit in RAM (1G) which the table does not: bench2=# select pg_size_pretty(pg_relation_size('pgbench_accounts_pkey')); pg_size_pretty 428 MB (1 row)

Re: [HACKERS] could not open relation with OID errors after promoting the standby to master

2012-05-16 Thread Alvaro Herrera
Excerpts from Joachim Wieland's message of mar may 15 22:37:15 -0400 2012: I've switched servers yesterday night and the previous slave is now the master. This is 9.0.6 (originally) / 9.0.7 (now) on Linux. Now I'm seeing a bunch of ERROR: could not open relation with OID 1990987633 #7

Re: [HACKERS] temporal support patch

2012-05-16 Thread Pavel Stehule
Hello what is conformance of your solution with temporal extension in ANSI SQL 2011 http://www.slideshare.net/CraigBaumunk/temporal-extensions-tosql20112012010438 Regards Pavel Stehule 2012/5/16 Miroslav Šimulčík simulcik.m...@gmail.com: Hi all, as a part of my master's thesis I have