Re: [HACKERS] [PATCH] Exorcise zero-dimensional arrays

2013-04-05 Thread Dimitri Fontaine
Brendan Jurd dire...@gmail.com writes: Well for what it's worth I would expect cardinality() to return the total number of elements in the array (per ArrayGetNItems). It's consistent with the spec's identification of an array as a collection. You can chunk the elements into dimensions

Re: [HACKERS] Clang compiler warning on 9.3 HEAD

2013-04-05 Thread Dimitri Fontaine
Alvaro Herrera alvhe...@2ndquadrant.com writes: Now, it annoys me that we now have three places that know about object types supported by event triggers: there's a large struct of command tag substrings (event_trigger_support), then there's these two functions. It might be better to add

Re: [HACKERS] CREATE EXTENSION BLOCKS

2013-04-05 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I think this should be addressed in extend.sgml not only on the CREATE EXTENSION reference page. After thinking awhile I came up with the attached wording. Further wordsmithing anyone? Thanks! -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL :

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Florian Pflug
On Apr4, 2013, at 23:21 , Jeff Janes jeff.ja...@gmail.com wrote: This brings up a pretty frightening possibility to me, unrelated to data checksums. If a bit gets twiddled in the WAL file due to a hardware issue or a cosmic ray, and then a crash happens, automatic recovery will stop early

Re: [HACKERS] Drastic performance loss in assert-enabled build in HEAD

2013-04-05 Thread Nicolas Barbier
2013/4/5 Noah Misch n...@leadboat.com: On Thu, Apr 04, 2013 at 12:28:01PM +0200, Nicolas Barbier wrote: +1. Having unlogged matviews without having incremental updates yet, isn't super useful anyway. I would have surmised the opposite: since an unlogged MV requires a full refresh at

Re: [HACKERS] Hash Join cost estimates

2013-04-05 Thread Matthias
In this example, hashing the large table is actually 2 seconds *faster* than hashing the small table (again, all on my laptop). Are you running the laptop on battery? When I've benchmarked pgsql last time I used my laptop as well and it only occured to me after a lot of trying that laptops

[HACKERS] Re: matview scannability rehash (was Re: Drastic performance loss in assert-enabled build in HEAD)

2013-04-05 Thread Noah Misch
On Thu, Apr 04, 2013 at 06:07:17PM -0400, Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Wed, Apr 03, 2013 at 05:49:18PM -0400, Tom Lane wrote: No. This is an absolute disaster. It's taking something we have always considered to be an irrelevant implementation detail and making

Re: [HACKERS] Hash Join cost estimates

2013-04-05 Thread Stephen Frost
* Matthias (nitrogen...@gmail.com) wrote: In this example, hashing the large table is actually 2 seconds *faster* than hashing the small table (again, all on my laptop). Are you running the laptop on battery? When I've benchmarked pgsql last time I used my laptop as well and it only occured

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Andres Freund
On 2013-04-04 17:39:16 -0700, Jeff Davis wrote: On Thu, 2013-04-04 at 22:39 +0200, Andres Freund wrote: I don't think its really slower. Earlier the code took WalInsertLock everytime, even if we ended up not logging anything. Thats far more epensive than a single spinlock. And the copy

Re: [HACKERS] [sepgsql 2/3] Add db_schema:search permission checks

2013-04-05 Thread Robert Haas
On Thu, Apr 4, 2013 at 8:26 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: OK, I follow the manner of the terminology as we usually call it. The attached patch just replaced things you suggested. Thanks, I have committed this, after making some changes to the comments and documentation. Please

Re: [HACKERS] matview scannability rehash (was Re: Drastic performance loss in assert-enabled build in HEAD)

2013-04-05 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: I realize that there's no other (easy) way to make unlogged matviews reset to an invalid state on crash, but that doesn't make this design choice less of a disaster.  It boxes us into something that's entirely unable to support transitions between scannable

Re: [HACKERS] Drastic performance loss in assert-enabled build in HEAD

2013-04-05 Thread Kevin Grittner
Noah Misch n...@leadboat.com wrote: On Thu, Apr 04, 2013 at 12:28:01PM +0200, Nicolas Barbier wrote: +1. Having unlogged matviews without having incremental updates yet, isn't super useful anyway. I would have surmised the opposite Hmm.  I was thinking about the fact that a full refresh can

Re: [HACKERS] matview scannability rehash (was Re: Drastic performance loss in assert-enabled build in HEAD)

2013-04-05 Thread Kevin Grittner
Noah Misch n...@leadboat.com wrote: The SQL commands I cited as responsible for creating or removing the fork all make a new relfilenode anyway.  Thus, add actually means creating the fork with the new relfilenode, and remove actually means omitting the fork from the new relfilenode.  The

[HACKERS] Re: matview scannability rehash (was Re: Drastic performance loss in assert-enabled build in HEAD)

2013-04-05 Thread Noah Misch
On Fri, Apr 05, 2013 at 07:00:38AM -0700, Kevin Grittner wrote: Noah Misch n...@leadboat.com wrote: The SQL commands I cited as responsible for creating or removing the fork all make a new relfilenode anyway.  Thus, add actually means creating the fork with the new relfilenode, and remove

Re: [HACKERS] matview scannability rehash (was Re: Drastic performance loss in assert-enabled build in HEAD)

2013-04-05 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: I think I need to review the whole thread again to make sure I wasn't too quick to concede the point. On a fresh reading of this, I think a large part of what is at issue here stems from a bad name for the new bool field I added to the RelationData

Re: [PATCH] Exorcise zero-dimensional arrays (Was: Re: [HACKERS] Should array_length() Return NULL)

2013-04-05 Thread Kevin Grittner
Brendan Jurd dire...@gmail.com wrote: The language specifically allows for zero elements, and does not contemplate multiple dimensions. I don't remember anything in the spec which would prohibit the data type of an array element from itself being an array, however. -- Kevin Grittner

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Jeff Davis
On Tue, 2013-03-26 at 03:34 +0200, Ants Aasma wrote: The main thing to look out for is that we don't have any blind spots for conceivable systemic errors. If we decide to go with the SIMD variant then I intend to figure out what the blind spots are and show that they don't matter. Are you

Re: [HACKERS] CREATE EXTENSION BLOCKS

2013-04-05 Thread David E. Wheeler
On Apr 4, 2013, at 2:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think this should be addressed in extend.sgml not only on the CREATE EXTENSION reference page. After thinking awhile I came up with the attached wording. Further wordsmithing anyone? Works for me, though I think it would be

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Ants Aasma
On Fri, Apr 5, 2013 at 7:23 PM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2013-03-26 at 03:34 +0200, Ants Aasma wrote: The main thing to look out for is that we don't have any blind spots for conceivable systemic errors. If we decide to go with the SIMD variant then I intend to figure out

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-05 Thread Robert Haas
On Fri, Apr 5, 2013 at 1:12 AM, Amit Kapila amit.kap...@huawei.com wrote: If we just put it to freelist, then next time if it get allocated directly from bufhash table, then who will remove it from freelist or do you think that, in BufferAlloc, if it gets from bufhash table, then it should

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Greg Smith
On 4/5/13 12:23 PM, Jeff Davis wrote: Are you still looking into SIMD? Right now, it's using the existing CRC implementation. Obviously we can't change it after it ships. Or is it too late to change it already? Simon just headed away for a break, so I'll try to answer this. He committed with

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Jeff Davis
On Fri, 2013-04-05 at 21:39 +0300, Ants Aasma wrote: Yes, I just managed to get myself some time so I can look at it some more. I was hoping that someone would weigh in on what their preferences are on the performance/effectiveness trade-off and the fact that we need to use assembler to make

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Kevin Grittner
Jeff Davis pg...@j-davis.com wrote: My opinion is that we don't need to be perfect as long as we catch 99% of random errors and we don't have any major blind spots. +1 Also, the first version doesn't necessarily need to perform well; we can leave optimization as future work. +1, as long

[HACKERS] unused code in float8_to_char , formatting.c ?

2013-04-05 Thread Greg Jaskiewicz
Hi Guys, Looking around the code Today, one of my helpful tools detected this dead code. As far as I can see, it is actually unused call to strlen() in formatting.c, float8_to_char(). Diff attached. formatting_dead_code.diff Description: Binary data -- GJ -- Sent via

[HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Tom Lane
After quite a bit of hair-pulling trying to install Fedora 19 Alpha, I've been able to reproduce the initdb-time failure that's currently being seen on buildfarm member anchovy, and was also complained of recently by an end user:

[HACKERS] Unrecognized type error (postgres 9.1.4)

2013-04-05 Thread Rodrigo Barboza
Hello. I created a type my_uint that is a unsigned int 32. I am trying to update data of a table that contains a column of this type. Here is what happens: postgresql= explain analyze UPDATE attribute_type_conf SET rowform = rowform +1 where rowform = 18; ERROR: unsupported type: 132852

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Peter Geoghegan
On Fri, Apr 5, 2013 at 11:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: It also strikes me that we ought to take this as a warning sign that we need to work on getting rid of coding like the above in favor of genuine flexible arrays, before the gcc boys think of some other overly-cute optimization

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Gavin Flower
On 06/04/13 11:14, Tom Lane wrote: After quite a bit of hair-pulling trying to install Fedora 19 Alpha, I've been able to reproduce the initdb-time failure that's currently being seen on buildfarm member anchovy, and was also complained of recently by an end user:

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: On Fri, Apr 5, 2013 at 11:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: It also strikes me that we ought to take this as a warning sign that we need to work on getting rid of coding like the above in favor of genuine flexible arrays, before the gcc boys

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Andres Freund
On 2013-04-05 23:28:03 +0100, Peter Geoghegan wrote: On Fri, Apr 5, 2013 at 11:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: It also strikes me that we ought to take this as a warning sign that we need to work on getting rid of coding like the above in favor of genuine flexible arrays, before

Re: [HACKERS] matview scannability rehash (was Re: Drastic performance loss in assert-enabled build in HEAD)

2013-04-05 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: I think a large part of what is at issue here stems from a bad name for the new bool field I added to the RelationData structure -- instead of rd_isscannable it should probably be called something like rd_ispopulated.  The current name led to some

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Peter Geoghegan
On Fri, Apr 5, 2013 at 11:39 PM, Andres Freund and...@2ndquadrant.com wrote: The respective macro magic is already in place, its just not used in all places. The problem is more that we can't easily use it in all places because e.g. in the one case mentioned here the array isn't in the last

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: The respective macro magic is already in place, its just not used in all places. The problem is more that we can't easily use it in all places because e.g. in the one case mentioned here the array isn't in the last place *in the back branches*. I

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: Are you proposing that we use the FLEXIBLE_ARRAY_MEMBER macro in every single place where we currently use the one element array pattern? Yup, exactly. I count one place where we currently use FLEXIBLE_ARRAY_MEMBER. It'd be pretty ugly to have that

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Peter Geoghegan
On Fri, Apr 5, 2013 at 11:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hm, I see 4 places in HEAD. But in any case, is My mistake. I had REL9_2_STABLE checked out. int16values[1];/* VARIABLE LENGTH ARRAY */ } int2vector; /* VARIABLE LENGTH STRUCT */

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jeff Davis
On Fri, 2013-04-05 at 15:09 +0200, Andres Freund wrote: How does the attached version look? I verified that it survives recovery, but not more. Comments: * Regarding full page writes, we can: - always write full pages (as in your current patch), regardless of the current settings - take

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jeff Davis
On Fri, 2013-04-05 at 10:34 +0200, Florian Pflug wrote: Maybe we could scan forward to check whether a corrupted WAL record is followed by one or more valid ones with sensible LSNs. If it is, chances are high that we haven't actually hit the end of the WAL. In that case, we could either log a

Re: [HACKERS] Drastic performance loss in assert-enabled build in HEAD

2013-04-05 Thread Noah Misch
On Fri, Apr 05, 2013 at 11:17:30AM +0200, Nicolas Barbier wrote: 2013/4/5 Noah Misch n...@leadboat.com: On Thu, Apr 04, 2013 at 12:28:01PM +0200, Nicolas Barbier wrote: +1. Having unlogged matviews without having incremental updates yet, isn't super useful anyway. I would have surmised

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jaime Casanova
On Fri, Apr 5, 2013 at 8:09 AM, Andres Freund and...@2ndquadrant.com wrote: How does the attached version look? I verified that it survives recovery, but not more. I still got errors when executing make installcheck in a just compiled 9.3devel + this_patch, this is when setting wal_level

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jeff Davis
On Fri, 2013-04-05 at 19:22 -0500, Jaime Casanova wrote: On Fri, Apr 5, 2013 at 8:09 AM, Andres Freund and...@2ndquadrant.com wrote: How does the attached version look? I verified that it survives recovery, but not more. I still got errors when executing make installcheck in a just

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jaime Casanova
On Fri, Apr 5, 2013 at 7:39 PM, Jeff Davis pg...@j-davis.com wrote: On Fri, 2013-04-05 at 19:22 -0500, Jaime Casanova wrote: On Fri, Apr 5, 2013 at 8:09 AM, Andres Freund and...@2ndquadrant.com wrote: How does the attached version look? I verified that it survives recovery, but not more.

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Peter Eisentraut
On Fri, 2013-04-05 at 18:14 -0400, Tom Lane wrote: Since gcc 4.8 is going to be on a lot of people's machines pretty soon, I think we need to do something to prevent it from breaking 8.4.x and 9.0.x. It looks like our choices are (1) teach configure to enable

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-05 Thread Peter Eisentraut
On Fri, 2013-04-05 at 23:44 +0100, Peter Geoghegan wrote: On Fri, Apr 5, 2013 at 11:39 PM, Andres Freund and...@2ndquadrant.com wrote: The respective macro magic is already in place, its just not used in all places. The problem is more that we can't easily use it in all places because e.g.

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-05 Thread Amit Kapila
On Saturday, April 06, 2013 12:38 AM Robert Haas wrote: On Fri, Apr 5, 2013 at 1:12 AM, Amit Kapila amit.kap...@huawei.com wrote: If we just put it to freelist, then next time if it get allocated directly from bufhash table, then who will remove it from freelist or do you think that, in

Re: [HACKERS] Unrecognized type error (postgres 9.1.4)

2013-04-05 Thread Amit Kapila
On Saturday, April 06, 2013 3:57 AM Rodrigo Barboza wrote: Hello. I created a type my_uint that is a unsigned int 32. I am trying to update data of a table that contains a column of this type. Here is what happens: postgresql= explain analyze UPDATE attribute_type_conf SET rowform =