Re: [HACKERS] Schema version management

2012-05-21 Thread Benedikt Grundmann
On Mon, May 21, 2012 at 5:03 AM, Joel Jacobson j...@trustly.com wrote: http://archives.postgresql.org/pgsql-hackers/2010-12/msg02301.php The initial feedback was on the usage of OIDs as file names. This was indeed a bad idea and was changed, see

[HACKERS] release note item

2012-05-21 Thread Andrew Dunstan
Regarding the item: * Properly handle empty arrays returned by PL/Perl functions (Andrew Dunstan) DETAILS? This was a bug fix, not a feature, and in any case is due to Alex Hunsaker, not me. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-21 Thread Robert Haas
On Sun, May 20, 2012 at 2:12 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Yeah, an enum would be nicer than an additional GUC. I kinda keep forgetting that we have those. Though to bikeshed, the GUC should probably be just called 'zero_pages' and take the values 'never', 'missing',

Re: [HACKERS] External Open Standards

2012-05-21 Thread Tom Lane
Brendan Jurd dire...@gmail.com writes: I'd be okay with just adding a note in the manual under Date/Time Output to the effect of Note: ISO 8601 specifies the use of uppercase letter 'T' to separate the date and time. Postgres uses a space for improved readability, in line with other database

Re: [HACKERS] release note item

2012-05-21 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Regarding the item: * Properly handle empty arrays returned by PL/Perl functions (Andrew Dunstan) DETAILS? This was a bug fix, not a feature, and in any case is due to Alex Hunsaker, not me. I assume this is in reference to these commits:

Re: [HACKERS] Archiver not exiting upon crash

2012-05-21 Thread Fujii Masao
On Sat, May 19, 2012 at 1:23 AM, Jeff Janes jeff.ja...@gmail.com wrote: I've been testing the crash recovery of REL9_2_BETA1, using the same method I posted in the Scaling XLog insertion thread.  I have the checkpointer occasionally throw a FATAL error, We should also fix this problem? If yes,

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, May 20, 2012 at 2:12 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Yeah, an enum would be nicer than an additional GUC. I kinda keep forgetting that we have those. Though to bikeshed, the GUC should probably be just called

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-21 Thread Robert Haas
On Mon, May 21, 2012 at 12:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: If we were sure that the kernel error was permanent, then this argument would be moot: the data is gone already.  The scary thought here is that it might be a transient error, such as a not-always-repeatable kernel bug.  In

Re: [HACKERS] Archiver not exiting upon crash

2012-05-21 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: ... sometimes the automatic recovery never initiates. It looks like the postmaster is waiting for the archiver to exit before it starts recovery, and the archiver is waiting for something, I don't really know what. Can you try poking into the

Re: [HACKERS] Archiver not exiting upon crash

2012-05-21 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: You might have gotten the following problem which was discussed before. This problem was fixed in SIGQUIT signal handler of a backend, but ISTM not that of an archiver. http://archives.postgresql.org/pgsql-admin/2009-11/msg00088.php pgarch.c's SIGQUIT

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, May 21, 2012 at 12:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: If we were sure that the kernel error was permanent, then this argument would be moot: the data is gone already.  The scary thought here is that it might be a transient error, such as

Re: [HACKERS] External Open Standards

2012-05-21 Thread Peter Eisentraut
On lör, 2012-05-19 at 11:52 -0400, Daniel Farina wrote: The documentation is misleading to the point of our support for ISO 8601-strict parsing. http://archives.postgresql.org/pgsql-hackers/2012-02/msg01237.php A very fine point, but I discovered it not out of curiosity, but a fairly

Re: [HACKERS] Archiver not exiting upon crash

2012-05-21 Thread Tom Lane
I wrote: Fujii Masao masao.fu...@gmail.com writes: You might have gotten the following problem which was discussed before. This problem was fixed in SIGQUIT signal handler of a backend, but ISTM not that of an archiver. http://archives.postgresql.org/pgsql-admin/2009-11/msg00088.php

Re: [HACKERS] External Open Standards

2012-05-21 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: The problem is that people think that ISO means ISO 8601, whereas it actually means ISO 9075. I can see how that's an easy mistake to make, though. ... especially since we keep referring to 8601 in our own docs. Does this mean we should do a global

Re: [HACKERS] Archiver not exiting upon crash

2012-05-21 Thread Tom Lane
I wrote: ... but having said that, I see Peter's commit d6de43099ac0bddb4b1da40088487616da892164 only touched postgres.c's quickdie(), and not all the *other* background processes with identical coding. That seems a clear oversight, so I will go fix it. Doesn't explain why the archiver

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Josh Berkus
Anyway, on my machine it seems that the per-tuple CPU costs for SELECT COUNT(*) with an index-only scan are something like 10% higher than the per-tuple costs with a heap scan. We might get that down to roughly par with some hacking, but it's never going to be vastly better. The argument

Re: [HACKERS] External Open Standards

2012-05-21 Thread Brendan Jurd
On 22 May 2012 02:58, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: The problem is that people think that ISO means ISO 8601, whereas it actually means ISO 9075.  I can see how that's an easy mistake to make, though. ... especially since we keep referring to

Re: [HACKERS] transformations between types and languages

2012-05-21 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Here is a draft design for the transforms feature, which I'd like to work on. ... fromsql takes one argument of the respective type and returns internal. tosql is the other way around. It's the responsibility of the language handler to look up this

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Well, if it's not CPU costs, then something else is eating the time, since I'm seeing per-tuple COUNT counts on indexes being 400% more than on heap. Well, I'm not: as I said, it looks like about 10% here. Perhaps you're testing a cassert-enabled build?

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Josh Berkus
On 5/21/12 10:41 AM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Well, if it's not CPU costs, then something else is eating the time, since I'm seeing per-tuple COUNT counts on indexes being 400% more than on heap. Well, I'm not: as I said, it looks like about 10% here. Perhaps

[HACKERS] heap metapages

2012-05-21 Thread Robert Haas
At dinner on Friday night at PGCon, the end of the table that included Tom Lane, Stephen Frost, and myself got to talking about the idea of including some kind of metapage in every relation, including heap relations. At least some index relations already have something like this (cf

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-21 Thread Robert Haas
On Mon, May 21, 2012 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, May 21, 2012 at 12:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: If we were sure that the kernel error was permanent, then this argument would be moot: the data is gone already.  

Re: [HACKERS] Draft release notes complete

2012-05-21 Thread Josh Berkus
For these reasons, it may be timely and appropriate, from a purely advocacy point-of-view, to call our new group commit group commit in release notes and documentation, and announce it as a new feature. First, shouldn't we be having this discussion on -advocacy? To date, I've been calling it

Re: [HACKERS] 9.2 Beta: intersection of daterange

2012-05-21 Thread Josh Berkus
On 5/16/12 1:40 AM, Brar Piening wrote: Misa Simic wrote: I think result is ok... 2010-01-04 is not inside first range... Staring at my query for five minutes obviously didn't prevent me from creating that noise. I meant to query SELECT '[2010-03-15,2010-05-22)'::daterange *

[HACKERS] A table for ALTER TABLE .. ENABLE TRIGGER combining session_replication_role GUC

2012-05-21 Thread Gurjeet Singh
A while ago I created this table to easily understand which kind of trigger gets executed under what setting of session_replication_role. There's already a description in the docs which describes it: quote Simply enabled triggers will fire when the replication role is origin(the default) or

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, May 21, 2012 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hm? zero_damaged_pages doesn't cause the buffer to be marked dirty, so I dunno where these alleged writes are coming from. I'm not sure either, but I'm pretty sure I've seen at

Re: [HACKERS] heap metapages

2012-05-21 Thread Merlin Moncure
On Mon, May 21, 2012 at 12:56 PM, Robert Haas robertmh...@gmail.com wrote: At dinner on Friday night at PGCon, the end of the table that included Tom Lane, Stephen Frost, and myself got to talking about the idea of including some kind of metapage in every relation, including heap relations.  

Re: [HACKERS] heap metapages

2012-05-21 Thread Robert Haas
On Mon, May 21, 2012 at 2:22 PM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, May 21, 2012 at 12:56 PM, Robert Haas robertmh...@gmail.com wrote: At dinner on Friday night at PGCon, the end of the table that included Tom Lane, Stephen Frost, and myself got to talking about the idea of

Re: [HACKERS] transformations between types and languages

2012-05-21 Thread Tom Lane
I wrote: Can we use something else for the magic type here? Or find a way to positively forbid such functions from being called from the SQL level? When I wrote that I was wondering if we'd need a new pg_proc column, prodontcallfromsql or some such. But on further reflection it seems like it

[HACKERS] Re: [BUGS] 9.2beta1 regression: pg_restore --data-only does not set sequence values any more

2012-05-21 Thread Andrew Dunstan
On 05/16/2012 10:23 AM, Andrew Dunstan wrote: On Wed, May 16, 2012 at 9:08 AM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us wrote: Martin Pitt mp...@debian.org mailto:mp...@debian.org writes: while packaging 9.2 beta 1 for Debian/Ubuntu the postgresql-common test

Re: [HACKERS] heap metapages

2012-05-21 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: The FSM and VM are small enough that interleaving them with the actual data probably wouldn't slow down seq scans materially. Wouldn't that end up potentially causing lots of random i/o if you need to look at many parts of the FSM or VM..? Also,

Re: [HACKERS] heap metapages

2012-05-21 Thread Simon Riggs
On 21 May 2012 13:56, Robert Haas robertmh...@gmail.com wrote: At dinner on Friday night at PGCon, the end of the table that included Tom Lane, Stephen Frost, and myself got to talking about the idea of including some kind of metapage in every relation, including heap relations.  At least

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Simon Riggs
On 21 May 2012 13:41, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: Well, if it's not CPU costs, then something else is eating the time, since I'm seeing per-tuple COUNT counts on indexes being 400% more than on heap. Well, I'm not: as I said, it looks like about

Re: [HACKERS] heap metapages

2012-05-21 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: The only thing against these ideas is that you're putting the design before the requirements, which always makes me nervous. [...] What springs immediately to mind is why this would not be just another fork. One of the requirements, though perhaps

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Surely the way to solve this is by having a new plan node that does a physical SeqScan of the index relation. It means we wouldn't preserve the sort order of the rows from the index, but that is just a plan cost issue. This is exactly what we do for

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Jeff Janes
On Mon, May 21, 2012 at 10:44 AM, Josh Berkus j...@agliodbs.com wrote: Right.  So what I'm trying to figure out is why counting an index which fits in ram (and I've confirmed via EXPLAIN ( buffers on ) ) is not being heap-fetched or read from disk would take 25% as long as counting a table

Re: [HACKERS] Schema version management

2012-05-21 Thread Daniel Farina
On Sun, May 20, 2012 at 9:03 PM, Joel Jacobson j...@trustly.com wrote: On Mon, May 21, 2012 at 10:06 AM, Daniel Farina dan...@heroku.com wrote: Also, now that I look more carefully, there was a lot of conversation about this patch; it seems like what you are doing now is reporting its

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Simon Riggs
On 21 May 2012 16:02, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Surely the way to solve this is by having a new plan node that does a physical SeqScan of the index relation. It means we wouldn't preserve the sort order of the rows from the index, but that is

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Josh Berkus
Earlier you said that this should be an ideal setup for IOS. But it isn't really--the ideal set up is one in which the alternative to an IOS is a regular index scan which makes many uncached scattered reads into the heap. I don't think that that situation can't really be engineered with a

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-21 Thread Heikki Linnakangas
On 18.05.2012 20:34, Alexander Korotkov wrote: On Fri, May 18, 2012 at 8:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: After fixing that, however, I'm now getting another error, much later in the build process: ERROR: failed to re-find parent for block 123002

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-21 Thread Heikki Linnakangas
On 18.05.2012 20:34, Alexander Korotkov wrote: On Fri, May 18, 2012 at 8:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: After fixing that, however, I'm now getting another error, much later in the build process: ERROR: failed to re-find parent for block 123002

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Simon Riggs
On 21 May 2012 16:42, Josh Berkus j...@agliodbs.com wrote: Earlier you said that this should be an ideal setup for IOS.  But it isn't really--the ideal set up is one in which the alternative to an IOS is a regular index scan which makes many uncached scattered reads into the heap.  I don't

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Jeff Janes
On Mon, May 21, 2012 at 1:42 PM, Josh Berkus j...@agliodbs.com wrote: Earlier you said that this should be an ideal setup for IOS.  But it isn't really--the ideal set up is one in which the alternative to an IOS is a regular index scan which makes many uncached scattered reads into the heap.  

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Merlin Moncure
On Mon, May 21, 2012 at 4:17 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, May 21, 2012 at 1:42 PM, Josh Berkus j...@agliodbs.com wrote: Earlier you said that this should be an ideal setup for IOS.  But it isn't really--the ideal set up is one in which the alternative to an IOS is a

Re: [HACKERS] read() returns ERANGE in Mac OS X

2012-05-21 Thread Florian Pflug
On May21, 2012, at 20:20 , Tom Lane wrote: I wonder whether we should dedicate a buffer status bit to show that the buffer has been zeroed by zero_damaged_pages and thus doesn't reflect what's on disk. Then we could teach autovacuum to not overwrite such pages. +1. The idea of us overwriting

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-21 Thread Alexander Korotkov
Hi! On Tue, May 22, 2012 at 12:56 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: After staring at graphs built from gist trees for the whole day, I think I finally understand what's wrong: There's a thinko in the way we maintain the parent paths during insertions. It

Re: [HACKERS] Patch: add conversion from pg_wchar to multibyte

2012-05-21 Thread Alexander Korotkov
Hello, Ishii-san! We've talked on PGCon that I've questions about mule to wchar conversion. My questions about pg_mule2wchar_with_len function are following. In these parts of code: * * else if (IS_LCPRV1(*from) len = 3) { from++; *to = *from++ 16; *to |= *from++; len -= 3; }

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Ants Aasma
On Tue, May 22, 2012 at 12:29 AM, Merlin Moncure mmonc...@gmail.com wrote: Generally though the real world wins (although the gains will be generally less spectacular) are heavily i/o bound queries where the indexed subset of data you want is nicely packed and the (non clustered) heap records

Re: [HACKERS] Schema version management

2012-05-21 Thread Joel Jacobson
On Tue, May 22, 2012 at 3:30 AM, Daniel Farina dan...@heroku.com wrote: Thank you, that's very informative.  I'd like to reiterate one question, though, which is something like: How do you feel that the since-committed directory-output/input support in pg_dump/pg_restore could or should

Re: [HACKERS] Schema version management

2012-05-21 Thread Andrew Dunstan
On 05/21/2012 08:25 PM, Joel Jacobson wrote: On Tue, May 22, 2012 at 3:30 AM, Daniel Farinadan...@heroku.com wrote: Thank you, that's very informative. I'd like to reiterate one question, though, which is something like: How do you feel that the since-committed directory-output/input

[HACKERS] How could we make it simple to access the log as a table?

2012-05-21 Thread Josh Berkus
Hackers, There's a lot of great information in the postgres logs. While we eventually want to get more sophisticated about providing users with status and history information, for 9.3 it would be really nice to just offer the current logs in system view format. Certainly this can be done ad-hoc

Re: [HACKERS] How could we make it simple to access the log as a table?

2012-05-21 Thread Stephen Frost
Josh, * Josh Berkus (j...@agliodbs.com) wrote: Certainly this can be done ad-hoc using CSV format and csv_fdw. However, such setups are fragile due to log rotation and other issues. It seems like we could come up with a better way. Ideas? This is really where I was hoping to eventually get

Re: [HACKERS] heap metapages

2012-05-21 Thread Robert Haas
On Mon, May 21, 2012 at 3:15 PM, Simon Riggs si...@2ndquadrant.com wrote: I very much like the idea of a common framework to support multiple requirements. If we can view a couple of other designs as well it may quickly become clear this is the right way. In any case, the topics discussed here

Re: [HACKERS] heap metapages

2012-05-21 Thread Robert Haas
On Mon, May 21, 2012 at 3:15 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: The FSM and VM are small enough that interleaving them with the actual data probably wouldn't slow down seq scans materially. Wouldn't that end up potentially causing lots of

Re: [HACKERS] Draft release notes complete

2012-05-21 Thread Noah Misch
On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: I have completed my draft of the 9.2 release notes, and committed it to git. Concerning Have psql \copy use libpq's SendQuery(), SendQuery() is a psql-internal interface, not a libpq interface. The array statistics patch added new

Re: [HACKERS] psql bug

2012-05-21 Thread Tom Lane
Teodor Sigaev teo...@sigaev.ru writes: Seems system() call cleanups sigaction state on FreeBSD. The root of problem is a threading library. In FreeBSD there are two versions of sigaction() (system() and others) depending on thread enabled. libpq library is compiled by default with

Re: [HACKERS] Strange query planner behavior

2012-05-21 Thread Tom Lane
Qi Huang huangq...@hotmail.com writes: Hi, hackers(I'm doing another project related to Postgres besides the gSOC, so this one is not for TABLESAMPLE) I have a query as follows :select *from affiliation a, author_aff_history his, author auwhere a.aff_name = his.aff_nameand

[HACKERS] Getting rid of cheap-startup-cost paths earlier

2012-05-21 Thread Tom Lane
Currently, the planner keeps paths that appear to win on the grounds of either cheapest startup cost or cheapest total cost. It suddenly struck me that in many simple cases (viz, those with no LIMIT, EXISTS, cursor fast-start preference, etc) we could know a-priori that cheapest startup cost is