Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Dimitri Fontaine
Josh Berkus j...@agliodbs.com writes: How *are* we detecting which version is installed, anyway? Is that in the pg_extenstions table? The installed version is in the pg_extenstion catalog, the version we're upgrading to is in the control file and can be seen in the system view

Re: [HACKERS] Compilation failed

2011-02-04 Thread Magnus Hagander
On Fri, Feb 4, 2011 at 06:19, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 4, 2011 at 12:11 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Feb 3, 2011 at 11:42 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Feb 3, 2011 at 11:02 PM, Fujii Masao masao.fu...@gmail.com wrote:

Re: [HACKERS] exposing COPY API

2011-02-04 Thread Itagaki Takahiro
field is shifted in your broken CSV file? For example, the case you find AB,CD,EF for 2 columns tables. I could provide a raw CSV reader for jagged files, but you still have to cook the returned fields into a proper tuple... -- Itagaki Takahiro jagged_csv_api-20110204.patch Description: Binary

Re: [HACKERS] SSI patch version 14

2011-02-04 Thread Heikki Linnakangas
On 02.02.2011 19:36, Kevin Grittner wrote: I really appreciate you putting this testing framework together. This is clearly the right way to do it, although we also clearly don't want this test in the check target at the root directory, because of the run time. It would be nice to get some

Re: [HACKERS] We need to log aborted autovacuums

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 12:59 AM, Josh Berkus j...@agliodbs.com wrote: Robert, Seeing as how there seem to be neither objections nor endorsements, I'm inclined to commit what I proposed more or less as-is.  There remains the issue of what do about the log spam.  Josh Berkus suggested logging

Re: [HACKERS] Compilation failed

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 6:39 AM, Fujii Masao masao.fu...@gmail.com wrote: When I ran make before make install, both successfully finished. So I guess that the make install rule in Makefile might be corrupted. Ah - that's the key. Not corrupted, but missing a dependency. Try it now... --

Re: [HACKERS] [GENERAL] Issues with generate_series using integer boundaries

2011-02-04 Thread Thom Brown
On 3 February 2011 13:58, Thom Brown t...@linux.com wrote: On 3 February 2011 13:32, Thom Brown t...@linux.com wrote: Actually, further testing indicates this causes other problems: postgres=# SELECT x FROM generate_series(1, 9,-1) AS a(x);  x ---  1 (1 row) Should return no rows.

Re: [HACKERS] Compilation failed

2011-02-04 Thread Fujii Masao
On Fri, Feb 4, 2011 at 9:07 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 4, 2011 at 6:39 AM, Fujii Masao masao.fu...@gmail.com wrote: When I ran make before make install, both successfully finished. So I guess that the make install rule in Makefile might be corrupted. Ah - that's

Re: [HACKERS] Compilation failed

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 8:07 AM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Feb 4, 2011 at 9:07 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 4, 2011 at 6:39 AM, Fujii Masao masao.fu...@gmail.com wrote: When I ran make before make install, both successfully finished. So I guess

Re: [HACKERS] SSI patch version 14

2011-02-04 Thread Kevin Grittner
Heikki Linnakangas wrote: On 02.02.2011 19:36, Kevin Grittner wrote: I really appreciate you putting this testing framework together. This is clearly the right way to do it, although we also clearly don't want this test in the check target at the root directory, because of the run time.

Re: [HACKERS] exposing COPY API

2011-02-04 Thread Andrew Dunstan
On 02/04/2011 05:49 AM, Itagaki Takahiro wrote: Here is a demonstration to support jagged input files. It's a patch on the latest patch. The new added API is: bool NextLineCopyFrom( [IN] CopyState cstate, [OUT] char ***fields, [OUT] int *nfields, [OUT] Oid *tupleOid) It

[HACKERS] SSI performance

2011-02-04 Thread Heikki Linnakangas
We know that the predicate locking introduced by the serializable snapshot isolation patch adds a significant amount of overhead, when it's used. It was fixed for sequential scans by acquiring a relation level lock upfront and skipping the locking after that, but the general problem for index

Re: [HACKERS] SSI performance

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 9:29 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The interesting thing is that CoarserLockCovers() accounts for 20% of the overall CPU time, or 2/3 of the overhead. The logic of PredicateLockAcquire is: 1. Check if we already have a lock on the

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Robert Haas
On Thu, Feb 3, 2011 at 4:24 PM, Josh Berkus j...@agliodbs.com wrote: So every package would include a script called upgrade.sql ( or upgrade.c? ) which is supposed to handle the upgrade, and it's up to the module author to power that, at least until 9.2?  Seem like the most reasonable course

Re: [HACKERS] Add ENCODING option to COPY

2011-02-04 Thread Hitoshi Harada
2011/2/4 Itagaki Takahiro itagaki.takah...@gmail.com: On Tue, Feb 1, 2011 at 13:08, Hitoshi Harada umi.tan...@gmail.com wrote: The third patch is attached, modifying mb routines so that they can receive conversion procedures as FmgrInof * and save the function pointer in CopyState. I tested

Re: [HACKERS] SQL/MED - file_fdw

2011-02-04 Thread Robert Haas
On Fri, Jan 21, 2011 at 8:12 AM, Shigeru HANADA han...@metrosystems.co.jp wrote: * Try strVal() instead of DefElem-val.str * FdwEPrivate seems too abbreviated for me. How about FileFdwPrivate? Thanks, fixed. Was there supposed to be a patch attached here? Or where is it? We are past out of

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I don't think we should commit something that for 9.1 that we may need to change incompatibly for 9.2. If we're not completely happy with it, it gets booted. Whatever we put in place here is going to be with us for a long, long time. So, what is it

Re: [HACKERS] REVIEW: PL/Python table functions

2011-02-04 Thread Hitoshi Harada
2011/1/28 Jan Urbański wulc...@wulczer.org: On 27/01/11 00:41, Jan Urbański wrote: I'm also attaching an updated version that should apply on top of my github refactor branch (or incrementally over the new set of refactor patches that I will post shortly to the refactor thread). Attached is

[HACKERS] CommitFest progress - or lack thereof

2011-02-04 Thread Robert Haas
With ten days left in the current CommitFest, being the last CommitFest for 9.1 development, there are presently 40 patches that are marked either Needs Review or Waiting on Author. The 11 patches that are Waiting on Author are the following: Synchronous Replication, transaction-controlled

Re: [HACKERS] SSI patch version 14

2011-02-04 Thread Heikki Linnakangas
On 04.02.2011 14:30, Kevin Grittner wrote: Heikki Linnakangas wrote: The assertion in RecordFreeIndexPage() is a reasonable sanity check, but I'm inclined to move it to the caller instead: I don't think the FSM should need to access predicate lock manager, even for an assertion. OK. So it

Re: [HACKERS] CommitFest progress - or lack thereof

2011-02-04 Thread Oleg Bartunov
Robert, I don't see btree_gist with knn-support. I'm afraid it'll be forgotten. Oleg On Fri, 4 Feb 2011, Robert Haas wrote: With ten days left in the current CommitFest, being the last CommitFest for 9.1 development, there are presently 40 patches that are marked either Needs Review or

Re: [HACKERS] Add ENCODING option to COPY

2011-02-04 Thread Tom Lane
Itagaki Takahiro itagaki.takah...@gmail.com writes: * Syntax: ENCODING encoding vs. ENCODING 'encoding' We don't have to quote encoding names in the patch, but we always need quotes for CREATE DATABASE WITH ENCODING. I think we should modify CREATE DATABASE to accept unquoted encoding names

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 10:13 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: I don't think we should commit something that for 9.1 that we may need to change incompatibly for 9.2.  If we're not completely happy with it, it gets booted.  Whatever we

Re: [HACKERS] CommitFest progress - or lack thereof

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 10:46 AM, Oleg Bartunov o...@sai.msu.su wrote: I don't see btree_gist with knn-support. I'm afraid it'll be forgotten. If you don't see it there, it's because you didn't add it. The deadline for getting your patch into the CommitFest application was January 15th, and

Re: [HACKERS] SSI performance

2011-02-04 Thread Heikki Linnakangas
On 04.02.2011 15:37, Robert Haas wrote: Not sure. How much benefit do we get from upgrading tuple locks to page locks? Should we just upgrade from tuple locks to full-relation locks? Hmm, good question. Page-locks are the coarsest level for the b-tree locks, but maybe that would make sense

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-04 Thread Robert Haas
On Fri, Jan 28, 2011 at 2:19 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2011/1/28 Robert Haas robertmh...@gmail.com: On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule pavel.steh...@gmail.com wrote: variant a) my first patch - detoast on first usage with avoiding to useless detoast

Re: [HACKERS] pl/python custom datatype parsers

2011-02-04 Thread Hitoshi Harada
2011/1/28 Jan Urbański wulc...@wulczer.org: On 23/12/10 15:15, Jan Urbański wrote: Here's a patch implementing custom parsers for data types mentioned in http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's an incremental patch on top of the plpython-refactor patch sent

Re: [HACKERS] Add ENCODING option to COPY

2011-02-04 Thread Hitoshi Harada
2011/2/5 Tom Lane t...@sss.pgh.pa.us: Itagaki Takahiro itagaki.takah...@gmail.com writes: * Syntax: ENCODING encoding vs. ENCODING 'encoding' We don't have to quote encoding names in the patch, but we always need quotes for CREATE DATABASE WITH ENCODING. I think we should modify CREATE

Re: [HACKERS] Add ENCODING option to COPY

2011-02-04 Thread Tom Lane
Hitoshi Harada umi.tan...@gmail.com writes: 2011/2/5 Tom Lane t...@sss.pgh.pa.us: The reason that we use quotes in CREATE DATABASE is that encoding names aren't assumed to be valid SQL identifiers.  If this patch isn't following the CREATE DATABASE precedent, it's the patch that's wrong, not

Re: [HACKERS] Add ENCODING option to COPY

2011-02-04 Thread Hitoshi Harada
2011/2/5 Tom Lane t...@sss.pgh.pa.us: Hitoshi Harada umi.tan...@gmail.com writes: 2011/2/5 Tom Lane t...@sss.pgh.pa.us: The reason that we use quotes in CREATE DATABASE is that encoding names aren't assumed to be valid SQL identifiers.  If this patch isn't following the CREATE DATABASE

Re: [HACKERS] SSI patch version 14

2011-02-04 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 04.02.2011 14:30, Kevin Grittner wrote: OK. So it looks like right now it will move to btvacuumpage(), right before the call to RecordFreeIndexPage(), and we will need to add it to one spot each in the GiST and GIN AMs, when

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread David E. Wheeler
On Feb 4, 2011, at 7:48 AM, Robert Haas wrote: I'd like to answer this question, but I have not had enough time to read through this patch in detail, because there are 97 patches in this CommitFest. The point I'm trying to make, however, is procedural. We shouldn't commit anything at the

Re: [HACKERS] Add ENCODING option to COPY

2011-02-04 Thread Tom Lane
Hitoshi Harada umi.tan...@gmail.com writes: 2011/2/5 Tom Lane t...@sss.pgh.pa.us: Yeah, putting backend-only stuff into that header is a nonstarter. Do you mean you think it' all right to define pg_cached_encoding_conversion() in pg_conversion_fn.h? That seems pretty random too. I still

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 11:48 AM, David E. Wheeler da...@kineticode.com wrote: Robert, I think that the core extension if pretty uncontroversial, modulo some minor issues. It's the upgrade process that's more controversial. I think the case can be made to accept even that part as Dim has

Re: [HACKERS] SSI performance

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 11:07 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 04.02.2011 15:37, Robert Haas wrote: Not sure.  How much benefit do we get from upgrading tuple locks to page locks?  Should we just upgrade from tuple locks to full-relation locks? Hmm, good

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread David E. Wheeler
On Feb 4, 2011, at 8:54 AM, Robert Haas wrote: Yeah, I understand. I believe Tom said he was going to look at the basic functionality with an eye toward commit, and I hope to look at it as well, either before or after it gets committed. Time permitting, I'd then like to look at this, but

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Dimitri Fontaine
David E. Wheeler da...@kineticode.com writes: Dim, I haven't followed that closely lately, but is the ALTER EXTENSION UPGRADE bit still a separate patch? Yes it is. It's an incremental that apply on top of the extension patch and get its own patch entry on the commit fest application:

Re: [HACKERS] CommitFest progress - or lack thereof

2011-02-04 Thread Oleg Bartunov
Aha, Teodor sent it to the list Dec 28, see http://archives.postgresql.org/message-id/4D1A1677.80300%40sigaev.ru After a month I didn't see any activity on this patch, so I I added it to https://commitfest.postgresql.org/action/patch_view?id=350 Jan 21 Now, I realised it was too late.

Re: [HACKERS] pl/python quoting functions

2011-02-04 Thread Hitoshi Harada
2011/1/11 Jan Urbański wulc...@wulczer.org: Here's a patch that adds a few PL/Python functions for quoting strings. It's an incremental patch on top of the plpython-refactor patch sent in http://archives.postgresql.org/message-id/4d135170.3080...@wulczer.org. Git branch for this patch:

[HACKERS] more buildfarm breakage

2011-02-04 Thread Robert Haas
mingw is unhappy with my latest stab at fixing the mess created by the errcodes patch last night. It appears that there are several files in src/port that include postgres.h even when FRONTEND is defined. For example, chklocale.c does this, which looks good: #ifndef FRONTEND #include postgres.h

Re: [HACKERS] SSI performance

2011-02-04 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The logic of PredicateLockAcquire is: 1. Check if we already have a lock on the tuple. 2. Check if we already have a lock on the page. 3. Check if we already have a lock on the relation. So if you're accessing a lot of rows,

Re: [HACKERS] multiset patch review

2011-02-04 Thread Robert Haas
On Mon, Jan 31, 2011 at 1:49 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: I removed collect() aggregate function because the result type is MULTISET in the spec. I keep all of other functions and operators because they won't break anything in the standard. When we will have true

Re: [HACKERS] arrays as pl/perl input arguments [PATCH]

2011-02-04 Thread Andrew Dunstan
On 02/03/2011 08:29 PM, Alex Hunsaker wrote: On Mon, Jan 31, 2011 at 01:34, Alexey Klyukinal...@commandprompt.com wrote: I've looked at the patch and added a test for arrays exceeding or equal maximum dimensions to check, whether the recursive function won't bring surprises there. I've

Re: [HACKERS] multiset patch review

2011-02-04 Thread Itagaki Takahiro
On Sat, Feb 5, 2011 at 02:29, Robert Haas robertmh...@gmail.com wrote: I am still not in favor of adding this syntax. I also don't like the syntax, but unfortunately, almost all of them are in the SQL standard :-(. In addition, Oracle already uses the same feature with the special syntax,

Re: [HACKERS] more buildfarm breakage

2011-02-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: mingw is unhappy with my latest stab at fixing the mess created by the errcodes patch last night. It appears that there are several files in src/port that include postgres.h even when FRONTEND is defined. For example, chklocale.c does this, which

Re: [HACKERS] multiset patch review

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 1:02 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Sat, Feb 5, 2011 at 02:29, Robert Haas robertmh...@gmail.com wrote: I am still not in favor of adding this syntax. I also don't like the syntax, but unfortunately, almost all of them are in the SQL standard

Re: [HACKERS] SSI performance

2011-02-04 Thread Kevin Grittner
I wrote: I just had a thought -- we already have the LocalPredicateLockHash HTAB to help with granularity promotion issues without LW locking. Offhand, I can't see any reason we couldn't use this for an initial check for a relation level lock, before going through the more rigorous pass

Re: [HACKERS] keeping a timestamp of the last stats reset (for a db, table and function)

2011-02-04 Thread Tomas Vondra
Dne 4.2.2011 03:37, Greg Smith napsal(a): Thinking I should start with why I think this patch is neat...most of the servers I deal with are up 24x7 minus small amounts of downtime, presuming everyone does their job right that is. In that environment, having a starting timestamp for when the

Re: [HACKERS] multiset patch review

2011-02-04 Thread Itagaki Takahiro
On Sat, Feb 5, 2011 at 03:04, Robert Haas robertmh...@gmail.com wrote: I am still not in favor of adding this syntax. I also don't like the syntax, but unfortunately, almost all of them are in the SQL standard :-(. The standard specifies this syntax for arrays, or for multisets? Multisets.

Re: [HACKERS] more buildfarm breakage

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 1:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree, that is not cool. OK, changed. BTW, I noted here that errcodes.h doesn't seem to get built till after src/port/ is built.  That seems wrong --- there is definitely code in there that needs to throw errors. It does,

Re: [HACKERS] multiset patch review

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 1:15 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Sat, Feb 5, 2011 at 03:04, Robert Haas robertmh...@gmail.com wrote: I am still not in favor of adding this syntax. I also don't like the syntax, but unfortunately, almost all of them are in the SQL standard

[HACKERS] SELECT ... WHERE fti_query ORDER BY numeric_col LIMIT x - problem

2011-02-04 Thread PostgreSQL - Hans-Jürgen Schönig
I have toyed around with KNN a little and I am pretty impressed when it comes to the results we have seen in the GIS world. Given the infrastructure we have at the moment I wonder if KNN can help to speedup queries like that: SELECT ... WHERE fti_query ORDER BY numeric_col LIMIT x The use case

[HACKERS] Linux filesystem performance and checkpoint sorting

2011-02-04 Thread Greg Smith
Switching to a new thread for this summary since there's some much more generic info here...at this point I've finished exploring the major Linux filesystem and tuning options I wanted to, as part of examining changes to the checkpoint code. You can find all the raw data at

Re: [HACKERS] Spread checkpoint sync

2011-02-04 Thread Greg Smith
Michael Banck wrote: On Sat, Jan 15, 2011 at 05:47:24AM -0500, Greg Smith wrote: For example, the pre-release Squeeze numbers we're seeing are awful so far, but it's not really done yet either. Unfortunately, it does not look like Debian squeeze will change any more (or has changed

Re: [HACKERS] arrays as pl/perl input arguments [PATCH]

2011-02-04 Thread Alex Hunsaker
On Fri, Feb 4, 2011 at 10:29, Andrew Dunstan and...@dunslane.net wrote: Anyone object to fixing the above as part of this patch? That is making plperl_(build_tuple_result, modify_tuple, return_next, hash_from_tuple) handle array and hash (composite/row) types consistently? And _that_ would be

Re: [HACKERS] Extensions support for pg_dump, patch v27

2011-02-04 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Actually, I was about to pick up and start working on the whole extensions patch series, but now I'm confused as to what and where is the latest version. Ok, here's what I have, attached, as patch v30. What

Re: [HACKERS] Add ENCODING option to COPY

2011-02-04 Thread Hitoshi Harada
2011/2/5 Tom Lane t...@sss.pgh.pa.us: Hitoshi Harada umi.tan...@gmail.com writes: 2011/2/5 Tom Lane t...@sss.pgh.pa.us: Yeah, putting backend-only stuff into that header is a nonstarter. Do you mean you think it' all right to define pg_cached_encoding_conversion() in pg_conversion_fn.h?

Re: [HACKERS] Spread checkpoint sync

2011-02-04 Thread Greg Smith
As already mentioned in the broader discussion at http://archives.postgresql.org/message-id/4d4c4610.1030...@2ndquadrant.com , I'm seeing no solid performance swing in the checkpoint sorting code itself. Better sometimes, worse others, but never by a large amount. Here's what the statistics

[HACKERS] Use a separate pg_depend.deptype for extension membership?

2011-02-04 Thread Tom Lane
The extensions patch currently records that an object is part of an extension by making a pg_depend entry with deptype 'i' (INTERNAL). While that has the behavior we want, I wonder whether it wouldn't be smarter in the long run to invent a new deptype for this purpose. We do not want people

Re: [HACKERS] multiset patch review

2011-02-04 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: On Fri, Feb 4, 2011 at 1:15 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: Multisets. But I chose the same function name and syntax because arrays *are* multisets by definition. In math class, maybe. But in programming, no. Multiset is a

Re: [HACKERS] Use a separate pg_depend.deptype for extension membership?

2011-02-04 Thread Andrew Dunstan
On 02/04/2011 02:08 PM, Tom Lane wrote: The extensions patch currently records that an object is part of an extension by making a pg_depend entry with deptype 'i' (INTERNAL). While that has the behavior we want, I wonder whether it wouldn't be smarter in the long run to invent a new deptype

Re: [HACKERS] Extensions support for pg_dump, patch v27

2011-02-04 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: What are the guc.c changes in this patch? They appear completely unrelated to the topic. Right. Didn't spot them. Sorry for the noise in the patch, it must be a merge problem in my repository. Do you want me to clean that up or is it already to late?

Re: [HACKERS] Use a separate pg_depend.deptype for extension membership?

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 2:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: The extensions patch currently records that an object is part of an extension by making a pg_depend entry with deptype 'i' (INTERNAL). While that has the behavior we want, I wonder whether it wouldn't be smarter in the long run

Re: [HACKERS] Spread checkpoint sync

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 2:08 PM, Greg Smith g...@2ndquadrant.com wrote: -The total number of buffers I'm computing based on the checkpoint writes being sorted it not a perfect match to the number reported by the checkpoint complete status line.  Sometimes they are the same, sometimes not.  Not

Re: [HACKERS] Use a separate pg_depend.deptype for extension membership?

2011-02-04 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: The extensions patch currently records that an object is part of an extension by making a pg_depend entry with deptype 'i' (INTERNAL). While that has the behavior we want, I wonder whether it wouldn't be smarter in the long run to invent a new deptype for

[HACKERS] OCLASS_FOREIGN_TABLE support is incomplete

2011-02-04 Thread Tom Lane
While poking at the extensions patch I noticed that doDeletion() and getObjectDescription() both seem to be missing cases for OCLASS_FOREIGN_TABLE. Surely this is broken. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] OCLASS_FOREIGN_TABLE support is incomplete

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 2:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: While poking at the extensions patch I noticed that doDeletion() and getObjectDescription() both seem to be missing cases for OCLASS_FOREIGN_TABLE.  Surely this is broken. I'll look into fixing this. -- Robert Haas

Re: [HACKERS] Use a separate pg_depend.deptype for extension membership?

2011-02-04 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: If we go with a new deptype, I was thinking of using 'm' (macro DEPENDENCY_MEMBER) but am not set on that. Have we been using any particular term to refer to the objects that belong to an extension? Do you

Re: [HACKERS] Extensions support for pg_dump, patch v27

2011-02-04 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: What are the guc.c changes in this patch? They appear completely unrelated to the topic. Right. Didn't spot them. Sorry for the noise in the patch, it must be a merge problem in my repository. Do you want

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-04 Thread Pavel Stehule
2011/2/4 Robert Haas robertmh...@gmail.com: On Fri, Jan 28, 2011 at 2:19 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2011/1/28 Robert Haas robertmh...@gmail.com: On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule pavel.steh...@gmail.com wrote: variant a) my first patch - detoast on first

Re: [HACKERS] Use a separate pg_depend.deptype for extension membership?

2011-02-04 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Dimitri Fontaine dimi...@2ndquadrant.fr writes: Do you really think the new dependency type has to be re-usable easily in the future? DEPENDENCY_EXTENSION ('e') would look fine by me. Hmm ... Haas suggested that too, but to me it seems confusing: which way

Re: [HACKERS] more buildfarm breakage

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 1:16 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 4, 2011 at 1:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree, that is not cool. OK, changed. It looks like pipe.c is full of backend-specific error-reporting code. I could rewrite it to work in the

Re: [HACKERS] Extensions support for pg_dump, patch v27

2011-02-04 Thread Tom Lane
While I'm looking at this ... what is the rationale for treating rewrite rules as members of extensions, ie, why does the patch touch rewriteDefine.c? ISTM a rule is a property of a table and could not sensibly be an independent member of an extension. If there is a use for that, why are table

Re: [HACKERS] Extensions support for pg_dump, patch v27

2011-02-04 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: While I'm looking at this ... what is the rationale for treating rewrite rules as members of extensions, ie, why does the patch touch rewriteDefine.c? ISTM a rule is a property of a table and could not sensibly be an independent member of an extension. If

Re: [HACKERS] Per-column collation, the finale

2011-02-04 Thread Bruce Momjian
Noah Misch wrote: On Thu, Feb 03, 2011 at 05:53:28PM +0200, Peter Eisentraut wrote: On tor, 2011-02-03 at 00:10 -0500, Noah Misch wrote: This is good stuff. I'll send you a new patch in a day or three for perhaps another round of performance tests. Some of the other issues

Re: [HACKERS] is_absolute_path incorrect on Windows

2011-02-04 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I have reviewed is_absolute_path() and have implemented path_is_relative_and_below_cwd() to cleanly handle cases like 'E:abc' on Win32; patch attached. This patch appears

Re: [HACKERS] Linux filesystem performance and checkpoint sorting

2011-02-04 Thread Josh Berkus
Greg, Thanks for doing these tests! So: Linux flavor? Kernel version? Disk system and PG directory layout? -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent

Re: [HACKERS] SSI performance

2011-02-04 Thread Kevin Grittner
I wrote: If this works, it would be a very minor change, which might eliminate a lot of that overhead for many common cases. With that change in place, I loaded actual data from one county for our most heavily searched table and searched it on the most heavily searched index. I returned

Re: [HACKERS] SSI performance

2011-02-04 Thread Kevin Grittner
I wrote: repeatable read [best] Time: 51.150 ms serializable [best] Time: 52.089 ms It occurred to me that taking the best time from each was likely to give a reasonable approximation of the actual overhead of SSI in this situation. That came out to about 1.8% in this (small) set of

Re: [HACKERS] Linux filesystem performance and checkpoint sorting

2011-02-04 Thread Mark Kirkwood
On 05/02/11 07:31, Greg Smith wrote: Switching to a new thread for this summary since there's some much more generic info here...at this point I've finished exploring the major Linux filesystem and tuning options I wanted to, as part of examining changes to the checkpoint code. You can find

Re: [HACKERS] Linux filesystem performance and checkpoint sorting

2011-02-04 Thread Stephen J. Butler
On Fri, Feb 4, 2011 at 12:31 PM, Greg Smith g...@2ndquadrant.com wrote: -Switching from ext3 to xfs gave over a 3X speedup on the smaller test set:  from the 600-700 TPS range to around 2200 TPS.  TPS rate on the larger data set actually slowed down a touch on XFS, around 10%.  Still, such a

[HACKERS] OpenVMS - an effort which needs guidance and support.

2011-02-04 Thread Bill Pedersen
I see there was significant discussion back in February 2010 as to an OpenVMS port of PostgreSQL. I am interested in seeing PostgreSQL ported to OpenVMS as it will then faciliate other Open Source ports - like OpenBravo. HP OpenVMS Engineering is encouraging Open Source ports. A set of machines

Re: [HACKERS] multiset patch review

2011-02-04 Thread Itagaki Takahiro
On Sat, Feb 5, 2011 at 04:24, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: In math class, maybe.  But in programming, no.  Multiset is a datatype.  Array is a different datatype.  There is no reason why we need to clutter our parser with extra keywords to support a non-standard feature

Re: [HACKERS] Named restore points

2011-02-04 Thread Jaime Casanova
On Tue, Feb 1, 2011 at 10:02 AM, Euler Taveira de Oliveira eu...@timbira.com wrote: Em 14-01-2011 17:41, Jaime Casanova escreveu: Here is a patch that implements named restore points. Sorry, I was swamped with work. :( Your patch no longer applied so I rebased it and slightly modified it.

Re: [HACKERS] Unexpected page allocation behavior on insert-only tables

2011-02-04 Thread Bruce Momjian
Tom Lane wrote: I wrote: In particular, now that there's a distinction between smgr flush and relcache flush, maybe we could associate targblock reset with smgr flush (only) and arrange to not flush the smgr level during ANALYZE --- basically, smgr flush would only be needed when

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-02-04 Thread Joachim Wieland
On Thu, Feb 3, 2011 at 11:46 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: I think we have 2 important technical issues here:  * The consistency is not perfect. Each transaction is started   with small delays in step 1, but we cannot guarantee no other   transaction between them.

Re: [HACKERS] compiler warning

2011-02-04 Thread Bruce Momjian
Magnus Hagander wrote: On Thu, Feb 3, 2011 at 04:40, Bruce Momjian br...@momjian.us wrote: I am seeing the following compiler warning for the past few days: ? ? ? ?basebackup.c:213: warning: variable `ptr' might be clobbered by ? ? ? ?`longjmp' or `vfork' and I see this comment in the

Re: [HACKERS] Linux filesystem performance and checkpoint sorting

2011-02-04 Thread Greg Smith
Mark Kirkwood wrote: Are you going to do some runs with ext4? I'd be very interested to see how it compares (assuming that you are on a kernel version 2.6.32 or later so ext4 is reasonably stable...). Yes, before I touch this system significantly I'll do ext4 as well, and this is running the

Re: [HACKERS] Linux filesystem performance and checkpoint sorting

2011-02-04 Thread Greg Smith
Josh Berkus wrote: So: Linux flavor? Kernel version? Disk system and PG directory layout? OS configuration and PostgreSQL settings are saved into the output from the later runs (I added that somewhere in the middle): http://www.2ndquadrant.us/pgbench-results/294/pg_settings.txt That's

Re: [HACKERS] ALTER TYPE 2: skip already-provable no-work rewrites

2011-02-04 Thread Robert Haas
On Thu, Jan 27, 2011 at 2:48 PM, Noah Misch n...@leadboat.com wrote: Done as attached.  This preserves compatibility with our current handling of composite type dependencies.  The rest you've seen before. OK, so I took a look at this in more detail today. The current logic for table rewrites

Re: [HACKERS] OCLASS_FOREIGN_TABLE support is incomplete

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 2:46 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 4, 2011 at 2:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: While poking at the extensions patch I noticed that doDeletion() and getObjectDescription() both seem to be missing cases for OCLASS_FOREIGN_TABLE.  

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-02-04 Thread Magnus Hagander
On Sat, Feb 5, 2011 at 04:50, Joachim Wieland j...@mcknight.de wrote: On Thu, Feb 3, 2011 at 11:46 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: It might be better to remove Windows-specific codes from the first try. I doubt Windows message queue is the best API in such console-based