Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Robert Haas
On Apr 16, 2011, at 1:48 AM, Greg Smith g...@2ndquadrant.com wrote: P.S. You know what else I feel should earn an automatic rejection without any reviewer even looking at the code? Greg is absolutely right. And to the two he listed, let me add another of my own gripes: failing to provide

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: Reading one's own diff and reflecting on what you've changed is one of the extremely underappreciated practices of good open-source software development. Minimizing the size of that diff is perhaps the most important thing someone can do in order to

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Greg Smith
Tom Lane wrote: * On the other side of the coin, I have seen many a patch that was written to minimize the length of the diff to the detriment of readability or maintainability of the resulting code, and that's *not* a good tradeoff. Sure. that's possible. But based on the reviews I've

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Greg Smith
Robert Haas wrote: The OP says that this patch maintains the WAL-before-data rule without any explanation of how it accomplishes that seemingly quite amazing feat. I assume I'm going to have to read this patch at some point to refute this assertion, and I think that sucks. I don't think

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Greg Stark
On Sat, Apr 16, 2011 at 7:24 AM, Robert Haas robertmh...@gmail.com wrote: The OP says that this patch maintains the WAL-before-data rule without any explanation of how it accomplishes that seemingly quite amazing feat.  I assume I'm going to have to read this patch at some point to refute

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-16 Thread Noah Misch
On Fri, Apr 15, 2011 at 03:01:04PM -0400, Tom Lane wrote: What is happening is that the preceding reindex table pg_index set the indcheckxmin flag for pg_index's indexes. The new table built by vacuum full contains no HOT chains at all, let alone broken ones, so at the end of reindex_index we

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Radosław Smogura
Greg Stark gsst...@mit.edu Saturday 16 April 2011 13:00:19 On Sat, Apr 16, 2011 at 7:24 AM, Robert Haas robertmh...@gmail.com wrote: The OP says that this patch maintains the WAL-before-data rule without any explanation of how it accomplishes that seemingly quite amazing feat. I assume I'm

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Marko Kreen
On Sat, Apr 16, 2011 at 8:48 AM, Greg Smith g...@2ndquadrant.com wrote: Joshua Berkus wrote: Guys, can we *please* focus on the patch for now, rather than the formatting, which is fixable with sed? Never, and that's not true.  Heikki was being nice; I wouldn't have even slogged through it

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: What he did, I gather, is treat the mmapped buffers as a read-only copy of the data. To actually make any modifications he copies it into shared buffers and treats them like normal. When the buffers get flushed from memory they get written and then the

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-16 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Fri, Apr 15, 2011 at 03:01:04PM -0400, Tom Lane wrote: What is happening is that the preceding reindex table pg_index set the indcheckxmin flag for pg_index's indexes. The new table built by vacuum full contains no HOT chains at all, let alone broken

[HACKERS] Bizarre reindex_relation API

2011-04-16 Thread Tom Lane
Why in the world is reindex_relation defined like this? #define REINDEX_CHECK_CONSTRAINTS 0x1 #define REINDEX_SUPPRESS_INDEX_USE 0x2 extern bool reindex_relation(Oid relid, bool toast_too, int flags); Seems like a rational design would have folded toast_too in as another flag bit,

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-16 Thread Noah Misch
On Sat, Apr 16, 2011 at 11:17:53AM -0400, Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Fri, Apr 15, 2011 at 03:01:04PM -0400, Tom Lane wrote: What is happening is that the preceding reindex table pg_index set the indcheckxmin flag for pg_index's indexes. The new table built by

Re: [HACKERS] switch UNLOGGED to LOGGED

2011-04-16 Thread Leonardo Francalanci
If the master crashes while a transaction that used CREATE TABLE is unfinished, both the master and the standby will indefinitely retain identical, stray (not referenced by pg_class) files. The catalogs do reference the relfilenode of each unlogged relation; currently, that

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-16 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Sat, Apr 16, 2011 at 11:17:53AM -0400, Tom Lane wrote: [ squint... ] Won't that result in the rebuild failing outright? We can't remove an index from the pending list till after it's rebuilt, for obvious reasons. That would be a problem if an ambuild

Re: [HACKERS] Bizarre reindex_relation API

2011-04-16 Thread Noah Misch
On Sat, Apr 16, 2011 at 11:52:47AM -0400, Tom Lane wrote: Why in the world is reindex_relation defined like this? #define REINDEX_CHECK_CONSTRAINTS 0x1 #define REINDEX_SUPPRESS_INDEX_USE0x2 extern bool reindex_relation(Oid relid, bool toast_too, int flags); Seems like a rational

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Peter Eisentraut
On Fri, 2011-04-15 at 12:32 +0200, Radosław Smogura wrote: I didn't included this, as diff, because of ~150kb size (mainly configure scripts, which are included in SVC). Due to this, You may download it from http://softperience.eu/downloads/pg_mmap_20110415.diff.bz2 (Legal: Work

Re: [HACKERS] Bizarre reindex_relation API

2011-04-16 Thread Robert Haas
On Apr 16, 2011, at 11:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Why in the world is reindex_relation defined like this? #define REINDEX_CHECK_CONSTRAINTS0x1 #define REINDEX_SUPPRESS_INDEX_USE0x2 extern bool reindex_relation(Oid relid, bool toast_too, int flags); Seems like a

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-16 Thread Joshua Berkus
All, Never, and that's not true. Heikki was being nice; I wouldn't have even slogged through it long enough to ask the questions he did before kicking it back as unusable. A badly formatted patch makes it impossible to evaluate whether the changes from a submission are reasonable or not

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Joshua Berkus
Radoslaw, I think 10% is quite good, as my stand-alone test of mmap vs. read shown that speed up of copying 100MB data to mem may be from ~20ms to ~100ms (depends on destination address). Of course deeper, system test simulating real usage will say more. In any case after good deals with

Re: [HACKERS] [COMMITTERS] pgsql: Rename pg_regress option --multibyte to --encoding

2011-04-16 Thread Andrew Dunstan
On 04/15/2011 04:36 PM, Peter Eisentraut wrote: On Fri, 2011-04-15 at 12:28 -0400, Andrew Dunstan wrote: Yeah, what is the point of this?: -# Adjust REGRESS_OPTS because we need a UTF8 database -REGRESS_OPTS = --dbname=$(CONTRIB_TESTDB) --multibyte=UTF8 --no-locale +# We need

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-16 Thread Greg Smith
Joshua Berkus wrote: Then you can say that politely and firmly with direct reference to the problem, rather than making the submitter feel bad. That's exactly what happened. And then you responded that it was possible to use a patch without fixing the formatting first. That's not true,

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Marko Kreen
On Fri, Apr 15, 2011 at 3:12 PM, Radosław Smogura rsmog...@softperience.eu wrote: On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote: The patch is quite hard to read because of random whitespace changes and other stylistic issues, but I have a couple of high-level questions on the

[HACKERS] ALTER TABLE INHERIT vs collations

2011-04-16 Thread Tom Lane
Right at the moment, ALTER INHERIT doesn't verify that collations match in a proposed inheritance child. So you can do this: regression=# create table foo (f1 text collate C); CREATE TABLE regression=# create table bar (f1 text collate POSIX); CREATE TABLE regression=# alter table bar inherit

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Radosław Smogura
Tom Lane t...@sss.pgh.pa.us Saturday 16 April 2011 17:02:32 Greg Stark gsst...@mit.edu writes: What he did, I gather, is treat the mmapped buffers as a read-only copy of the data. To actually make any modifications he copies it into shared buffers and treats them like normal. When the

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-16 Thread Thom Brown
On 16 April 2011 23:23, Tom Lane t...@sss.pgh.pa.us wrote: Right at the moment, ALTER INHERIT doesn't verify that collations match in a proposed inheritance child.  So you can do this: regression=# create table foo (f1 text collate C); CREATE TABLE regression=# create table bar (f1 text

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Greg Smith
Radosław Smogura wrote: Yes, but, hmm... in Netbeans I had really long gaps (probably 8 spaces, from tabs), so deeper ifs, comments at the and of variables, went of out my screen. I really wanted to not format this, but sometimes I needed. The guide at

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Tom Lane
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes: No, no, no :) I wanted to do this, but from above reason I skipped it. I swap VM pages, I do remap, in place where the shared buffer was I put mmaped page, and in place where mmaped page was I put shared page (in certain

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-16 Thread Tom Lane
Thom Brown t...@linux.com writes: On 16 April 2011 23:23, Tom Lane t...@sss.pgh.pa.us wrote: Does anyone think it's not a bug that ALTER TABLE lets this through? If so, what do you think the querying semantics ought to be? An argument to not treat it as a bug might be to suggest that the

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-16 Thread Thom Brown
On 17 April 2011 00:40, Tom Lane t...@sss.pgh.pa.us wrote: Thom Brown t...@linux.com writes: On 16 April 2011 23:23, Tom Lane t...@sss.pgh.pa.us wrote: Does anyone think it's not a bug that ALTER TABLE lets this through? If so, what do you think the querying semantics ought to be? An

[HACKERS] WIP patch for allowing COLLATE in plpgsql variable declarations

2011-04-16 Thread Tom Lane
The attached code patch (seems to work, but no regression tests or doc fixes yet) allows plpgsql variable declarations like declare x text collate en_US; Such a collation spec overrides the default variable collation that is otherwise inherited from the function's input

[HACKERS] Windows 64 bit warnings

2011-04-16 Thread Andrew Dunstan
While looking into setting up some libraries to use for 64 bit Windows builds, I took a quick look at the output from the 64 bit postgres builds currently running. They're actually quite clean, a heck of a lot cleaner than several other packages I have been looking at, quite a good testament

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-16 Thread Christopher Browne
On Sat, Apr 16, 2011 at 3:19 PM, Greg Smith g...@2ndquadrant.com wrote: Joshua Berkus wrote: Then you can say that politely and firmly with direct reference to the problem, rather than making the submitter feel bad. That's exactly what happened.  And then you responded that it was possible

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-16 Thread Robert Haas
On Sat, Apr 16, 2011 at 6:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Right at the moment, ALTER INHERIT doesn't verify that collations match in a proposed inheritance child.  So you can do this: regression=# create table foo (f1 text collate C); CREATE TABLE regression=# create table bar (f1

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-16 Thread Robert Haas
On Sat, Apr 16, 2011 at 9:12 PM, Christopher Browne cbbro...@gmail.com wrote: On Sat, Apr 16, 2011 at 3:19 PM, Greg Smith g...@2ndquadrant.com wrote: Joshua Berkus wrote: Then you can say that politely and firmly with direct reference to the problem, rather than making the submitter feel bad.

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-16 Thread Andrew Dunstan
On 04/16/2011 09:12 PM, Christopher Browne wrote: On Sat, Apr 16, 2011 at 3:19 PM, Greg Smithg...@2ndquadrant.com wrote: Joshua Berkus wrote: Then you can say that politely and firmly with direct reference to the problem, rather than making the submitter feel bad. That's exactly what

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Robert Haas
On Sat, Apr 16, 2011 at 2:33 PM, Joshua Berkus j...@agliodbs.com wrote: Well, given the risks to durability and stability associated with using MMAP, I doubt anyone would even consider it for a 10% throughput improvement.   However, I don't think the test you used demonstrates the best case

Re: [HACKERS] MMAP Buffers

2011-04-16 Thread Robert Haas
On Sat, Apr 16, 2011 at 9:31 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Apr 16, 2011 at 2:33 PM, Joshua Berkus j...@agliodbs.com wrote: Well, given the risks to durability and stability associated with using MMAP, I doubt anyone would even consider it for a 10% throughput

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-16 Thread Jean-Pierre Pelletier
One use case for this might be with constraint exclusion where there would be one partition per collation (language) with queries against the parent table always being for exactly one language. Not sure what the collation should be in the parent table then. -- Sent via pgsql-hackers mailing

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-16 Thread Kevin Grittner
Andrew Dunstan wrote: What makes you think this isn't possible to run pgindent? I have to say, I've been rather mystified by the difficulty attributed to running pgindent. During work on the SSI patch, I ran it about once every two weeks on files involved in the patch, just so that it would