Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Fujii Masao
On Fri, Jan 20, 2012 at 1:01 PM, Steve Singer ssinger...@sympatico.ca wrote: Here is my review of this verison of the patch. I think this patch has been in every CF for 9.2 and I feel it is getting close to being committed. Thanks for the review! Testing Review

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Erik Rijkers
On Fri, January 20, 2012 05:01, Steve Singer wrote: On 12-01-17 05:38 AM, Fujii Masao wrote: On Fri, Jan 13, 2012 at 5:02 PM, Fujii Masaomasao.fu...@gmail.com wrote: The amount of code changes to allow pg_basebackup to make a backup from the standby seems to be small. So I ended up merging

Re: [HACKERS] WAL Restore process during recovery

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 3:43 AM, Fujii Masao masao.fu...@gmail.com wrote: Requested update --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Support, Training Services diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c

Re: [HACKERS] WAL Restore process during recovery

2012-01-20 Thread Fujii Masao
On Fri, Jan 20, 2012 at 7:38 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Jan 20, 2012 at 3:43 AM, Fujii Masao masao.fu...@gmail.com wrote: Requested update Thanks! Will review. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center --

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Fujii Masao
On Fri, Jan 20, 2012 at 7:37 PM, Erik Rijkers e...@xs4all.nl wrote: I'm not sure, but it does look like this is the mystery bug that I encountered repeatedly already in 9.0devel; but I was never able to reproduce it reliably.  But I don't think it was ever solved.  

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Simon Riggs
On Tue, Jan 17, 2012 at 10:38 AM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Jan 13, 2012 at 5:02 PM, Fujii Masao masao.fu...@gmail.com wrote: The amount of code changes to allow pg_basebackup to make a backup from the standby seems to be small. So I ended up merging that changes and the

Re: [HACKERS] IDLE in transaction introspection

2012-01-20 Thread Magnus Hagander
On Thu, Jan 19, 2012 at 15:42, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Jan 19, 2012 at 10:31 PM, Magnus Hagander mag...@hagander.net wrote: Applied with fairly extensive modifications. I moved things around, switched to using enum instead of int+#define and a few things like that.

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 11:04 AM, Fujii Masao masao.fu...@gmail.com wrote: But Steve encountered it again, which means that the above fix is not sufficient. Unless the issue is derived from my patch, we should do another cycle of diagnosis of it. It's my bug, and I've posted a fix but not yet

Re: [HACKERS] New replication mode: write

2012-01-20 Thread Simon Riggs
On Mon, Jan 16, 2012 at 4:17 PM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, Jan 16, 2012 at 12:45 PM, Fujii Masao masao.fu...@gmail.com wrote: Done. Attached is the updated version of the patch. Thanks. I'll review this first, but can't start immediately. Please expect something back

Re: [HACKERS] PATCH: tracking temp files in pg_stat_database

2012-01-20 Thread Magnus Hagander
On Tue, Jan 17, 2012 at 21:39, Tomas Vondra t...@fuzzy.cz wrote: On 20.12.2011 19:59, Tomas Vondra wrote: On 20.12.2011 11:20, Magnus Hagander wrote: 2011/12/20 Tomas Vondra t...@fuzzy.cz: I haven't updated the docs yet - let's see if the patch is acceptable at all first. Again, without

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Fujii Masao
Thanks for the review! On Fri, Jan 20, 2012 at 8:15 PM, Simon Riggs si...@2ndquadrant.com wrote: I'm looking at this patch and wondering why we're doing so many press-ups to ensure full_page_writes parameter is on. This will still fail if you use a utility that removes the full page writes,

Re: [HACKERS] pg_basebackup is not checking IDENTIFY_SYSTEM numbre of columns

2012-01-20 Thread Magnus Hagander
On Sun, Jan 15, 2012 at 22:00, Jaime Casanova ja...@2ndquadrant.com wrote: On Wed, Jan 11, 2012 at 5:11 PM, Magnus Hagander mag...@hagander.net wrote: No, no reason. Adding such a check would be a good idea. ok. patch attached, it also adds a few PQclear() calls before

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-01-20 Thread Robert Haas
On Sat, Jan 14, 2012 at 9:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Here's another version of the patch to make XLogInsert less of a bottleneck on multi-CPU systems. The basic idea is the same as before, but several bugs have been fixed, and lots of misc. clean up has

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: I've taken that idea and used it to build a second Clog cache, known as ClogHistory which allows access to the read-only tail of pages in the clog. Once a page has been written to for the last time, it will be accessed

Re: [HACKERS] Inline Extension

2012-01-20 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Robert Haas robertmh...@gmail.com writes: I guess the question is: for what purpose? Indeed, it seems like such a thing is not an extension at all anymore, or at least it gives up many of the useful properties of extensions. I'm thinking that a common name

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:07 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 01/19/2012 04:12 PM, Robert Haas wrote: On Thu, Jan 19, 2012 at 4:07 PM, Andrew Dunstanand...@dunslane.net  wrote: The spec only allows unescaped Unicode chars (and for our

Re: [HACKERS] Command Triggers

2012-01-20 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: My advice is to forget about trying to provide the command string to the user for the first version of this patch. As you're finding out, there's no simple, easy, obvious way of doing it, and there are N0 useful things that can be done without that

Re: [HACKERS] our buffer replacement strategy is kind of lame

2012-01-20 Thread Heikki Linnakangas
On 03.01.2012 17:56, Simon Riggs wrote: On Tue, Jan 3, 2012 at 3:18 PM, Robert Haasrobertmh...@gmail.com wrote: 2. When a backend can't find a free buffer, it spins for a long time while holding the lock. This makes the buffer strategy O(N) in its worst case, which slows everything down.

Re: [HACKERS] pg_basebackup option for handling symlinks

2012-01-20 Thread Magnus Hagander
On Mon, Jan 16, 2012 at 19:52, Peter Eisentraut pete...@gmx.net wrote: On sön, 2012-01-08 at 22:22 +0100, Magnus Hagander wrote: On Sun, Jan 8, 2012 at 21:53, Peter Eisentraut pete...@gmx.net wrote: I've recently had a possible need for telling pg_basebackup how to handle symlinks in the

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 12:54 PM, Fujii Masao masao.fu...@gmail.com wrote: Thanks for the review! On Fri, Jan 20, 2012 at 8:15 PM, Simon Riggs si...@2ndquadrant.com wrote: I'm looking at this patch and wondering why we're doing so many press-ups to ensure full_page_writes parameter is on.

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 2:11 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 20.01.2012 15:32, Robert Haas wrote: On Sat, Jan 14, 2012 at 9:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com  wrote: Here's another version of the patch to make XLogInsert less

Re: [HACKERS] Inline Extension

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 8:52 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Tom Lane t...@sss.pgh.pa.us writes: Robert Haas robertmh...@gmail.com writes: I guess the question is: for what purpose? Indeed, it seems like such a thing is not an extension at all anymore, or at least it gives

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-20 Thread Marko Kreen
On Tue, Jan 17, 2012 at 05:53:33PM +0900, Kyotaro HORIGUCHI wrote: Hello, This is revised and rebased version of the patch. a. Old term `Add Tuple Function' is changed to 'Store Handler'. The reason why not `storage' is simply length of the symbols. b. I couldn't find the place to

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: I've taken that idea and used it to build a second Clog cache, known as ClogHistory which allows access to the read-only tail of pages in the clog.

Re: [HACKERS] our buffer replacement strategy is kind of lame

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 9:29 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I'd like to see some benchmarks that show a benefit from these patches, before committing something like this that complicates the code. These patches are fairly small, but nevertheless. Once we have

Re: [HACKERS] Patch review for logging hooks (CF 2012-01)

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 2:47 AM, Greg Smith g...@2ndquadrant.com wrote: The updated patch looks good, marking as 'Ready for Committer' Patches without documentation are never ready for commit.  For this one, I'm not sure if that should be in the form of a reference example in contrib, or just

Re: [HACKERS] Command Triggers

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 9:28 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: My advice is to forget about trying to provide the command string to the user for the first version of this patch.  As you're finding out, there's no simple, easy, obvious

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 9:44 AM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: I've taken that idea and used it to build a second Clog cache, known as

Re: [HACKERS] Patch review for logging hooks (CF 2012-01)

2012-01-20 Thread Marti Raudsepp
On Fri, Jan 20, 2012 at 17:01, Robert Haas robertmh...@gmail.com wrote: We should probably at least have a working example for testing purposes, though, whether or not we end up committing it. Martin Pihlak sent a short description of how to test the patch with his pg_logforward module:

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The code I've written so far does no canonicalization of the input value of any kind, just as we do for XML. Fair enough. So, given that framework, what the patch does is this: if you're using UTF-8, then \u is accepted, provided that is

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:46 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 19 January 2012 17:40, Robert Haas robertmh...@gmail.com wrote: I don't know what you mean by this.  I think removing wal_writer_delay is premature, because I think it still may have some utility, and the patch

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The code I've written so far does no canonicalization of the input value of any kind, just as we do for XML. Fair enough. So, given that framework, what the patch does is this: if

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: I've taken that idea and used it to build a second Clog cache, known as

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-20 Thread Robert Haas
On Thu, Jan 19, 2012 at 11:29 PM, Greg Smith g...@2ndquadrant.com wrote: I chewed a bit on Heikki's comment that similarity to the query planning parameters might be useful, and Robert's that being able to explain how the feature works more easily has value.  I have an initial adjustment of my

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 3:32 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote:

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:38 AM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Jan 20, 2012 at 3:32 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas robertmh...@gmail.com

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Andrew Dunstan
On 01/20/2012 09:19 AM, Robert Haas wrote: On Fri, Jan 20, 2012 at 12:07 AM, Tom Lanet...@sss.pgh.pa.us wrote: Andrew Dunstanand...@dunslane.net writes: On 01/19/2012 04:12 PM, Robert Haas wrote: On Thu, Jan 19, 2012 at 4:07 PM, Andrew Dunstanand...@dunslane.netwrote: The spec only

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: I've taken that idea and used it to build a second Clog cache, known as ClogHistory which allows access to the read-only tail of pages in the clog.

Re: [HACKERS] Inline Extension

2012-01-20 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: peeves is that the system doesn't know how to do an install of v1.1 by running the v1.0 script followed by the 1.0-1.1 upgrade script, which Did you try CREATE EXTENSION foo FROM 1.0; Maybe you would want the system to be able to determine the

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Marti Raudsepp
On Fri, Jan 20, 2012 at 01:33, Erik Rijkers e...@xs4all.nl wrote: Btw, it seems impossible to Ctrl-C out of a search once it is submitted; I suppose this is normally necessary for perfomance reasons, but it would be useful te be able to compile a test version that allows it. I believe

Re: [HACKERS] Inline Extension

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 11:29 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: peeves is that the system doesn't know how to do an install of v1.1 by running the v1.0 script followed by the 1.0-1.1 upgrade script, which Did you try  CREATE

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:45 AM, Andrew Dunstan and...@dunslane.net wrote: XML's #; escape mechanism is more or less the equivalent of JSON's \u. But XML documents can be encoded in a variety of encodings, including non-unicode encodings such as Latin-1. However, no matter what the

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread David E. Wheeler
On Jan 19, 2012, at 9:07 PM, Tom Lane wrote: If his client encoding is UTF8, the value will be letter-perfect JSON when it gets to him; and if his client encoding is not UTF8, then he's already pretty much decided that he doesn't give a fig about the Unicode-centricity of the JSON spec, no?

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread David E. Wheeler
On Jan 20, 2012, at 8:58 AM, Robert Haas wrote: If, however, we're not using UTF-8, we have to first turn \u into a Unicode code point, then covert that to a character in the database encoding, and then test for equality with the other character after that. I'm not sure whether that's

Re: [HACKERS] Command Triggers

2012-01-20 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I think the OID is better than the name, but if it's easy to pass the name and schema, then I'm fine with it. But I do think this is one of It's quite easy to get name and schema from the command yes, here's an example of how I'm doing it for some

Re: [HACKERS] Remembering bug #6123

2012-01-20 Thread Kevin Grittner
Kevin Grittner kevin.gritt...@wicourts.gov wrote: Tom Lane wrote: Well, the bottom line that's concerning me here is whether throwing errors is going to push anyone's application into an unfixable corner. I'm somewhat encouraged that your Circuit Courts software can adapt to it, since

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Garick Hamlin
On Fri, Jan 20, 2012 at 09:12:13AM -0800, David E. Wheeler wrote: On Jan 19, 2012, at 9:07 PM, Tom Lane wrote: If his client encoding is UTF8, the value will be letter-perfect JSON when it gets to him; and if his client encoding is not UTF8, then he's already pretty much decided that he

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Andrew Dunstan
On 01/20/2012 11:58 AM, Robert Haas wrote: On Fri, Jan 20, 2012 at 10:45 AM, Andrew Dunstanand...@dunslane.net wrote: XML's#; escape mechanism is more or less the equivalent of JSON's \u. But XML documents can be encoded in a variety of encodings, including non-unicode encodings such

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-20 Thread Greg Smith
On 01/20/2012 10:37 AM, Robert Haas wrote: On Thu, Jan 19, 2012 at 11:29 PM, Greg Smithg...@2ndquadrant.com wrote: vacuum_cost_page_hit = 0.1 vacuum_cost_page_miss = 1.0 vacuum_cost_page_dirty = 2.0 Now add in the new setting, which is explicitly said to be the read value:

Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-20 Thread Heikki Linnakangas
On 04.01.2012 13:14, Simon Riggs wrote: On Tue, Jan 3, 2012 at 11:28 PM, Tom Lanet...@sss.pgh.pa.us wrote: Jim Nasbyj...@nasby.net writes: On Jan 3, 2012, at 12:11 PM, Simon Riggs wrote: This could well be related to the fact that DropRelFileNodeBuffers() does a scan of shared_buffers,

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:14 PM, David E. Wheeler da...@kineticode.com wrote: On Jan 20, 2012, at 8:58 AM, Robert Haas wrote: If, however, we're not using UTF-8, we have to first turn \u into a Unicode code point, then covert that to a character in the database encoding, and then test

Re: [HACKERS] Command Triggers

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:14 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: I think the OID is better than the name, but if it's easy to pass the name and schema, then I'm fine with it.  But I do think this is one of It's quite easy to get name

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:35 PM, Greg Smith g...@2ndquadrant.com wrote: On 01/20/2012 10:37 AM, Robert Haas wrote: On Thu, Jan 19, 2012 at 11:29 PM, Greg Smithg...@2ndquadrant.com  wrote: vacuum_cost_page_hit = 0.1 vacuum_cost_page_miss = 1.0 vacuum_cost_page_dirty = 2.0 Now add in the

Re: [HACKERS] Inline Extension

2012-01-20 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Maybe you would want the system to be able to determine the oldest version to start from to reach the current default_version given in the control file, but I guess it would be better to add another property like default_full_version or such

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Err, actually, now that I think about it, that might be a problem: what happens if we're trying to test two characters for equality and the encoding conversion fails? This is surely all entirely doable given the encoding infrastructure we already have.

Re: [HACKERS] Command Triggers

2012-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Hmm, OK. But what happens if the user doesn't specify a schema name explicitly? For the creation case, RangeVarGetCreationNamespace should handle that. The code Dimitri quoted is wrong, but not that hard to fix. Unfortunately, the code he quoted for

Re: [HACKERS] Command Triggers

2012-01-20 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: For the creation case, RangeVarGetCreationNamespace should handle that. The code Dimitri quoted is wrong, but not that hard to fix. Ok. Unfortunately, the code he quoted for the ALTER case is also wrong, and harder to fix. Until you've done the lookup you

Re: [HACKERS] Checkpoint sync pause

2012-01-20 Thread Robert Haas
On Mon, Jan 16, 2012 at 8:59 PM, Greg Smith g...@2ndquadrant.com wrote: [ interesting description of problem scenario and necessary conditions for reproducing it ] This is about what I thought was happening, but I'm still not quite sure how to recreate it in the lab. Have you had a chance to

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Err, actually, now that I think about it, that might be a problem: what happens if we're trying to test two characters for equality and the encoding conversion fails? This is surely

Re: [HACKERS] Remembering bug #6123

2012-01-20 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: After a couple meetings, I have approval to get this into an application release currently in development. Assuming that your patch from the 13th is good for doing the testing, I think I can post test results in about three weeks. I'll also

Re: [HACKERS] Remembering bug #6123

2012-01-20 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: I'm up to my elbows in planner guts at the moment, but will try to fix up the patch this weekend if you want. They have scheduled testers to check on this issue next week, so it would be great to get as close as we can on the stuff that matters. -Kevin

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Thu, Jan 19, 2012 at 8:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Frankly I don't see the point of this. If the extension is an independent piece of (SQL) code, developed separately from an application, with its own lifecycle, a

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Heikki Linnakangas
I spent some time cleaning this up. Details below, but here are the highlights: * Reverted the removal of wal_writer_delay * Doesn't rely on WAL writer. Any process can act as the leader now. * Removed heuristic on big flushes * Uses PGSemaphoreLock/Unlock instead of latches On 20.01.2012

Re: [HACKERS] Inline Extension

2012-01-20 Thread Heikki Linnakangas
On 21.01.2012 00:00, Daniel Farina wrote: I think this is somewhat rube-goldberg-esque, and denies non-superuser roles the ability to get more version management of schema and operators. As-is many organizations are submitting migrations via plain SQL that include committing to a version

Re: [HACKERS] gistVacuumUpdate

2012-01-20 Thread Heikki Linnakangas
On 18.01.2012 23:38, YAMAMOTO Takashi wrote: i'm wondering because what gistVacuumUpdate used to do does not seem to be necessarily tied to the old-style VACUUM FULL. currently, no one will re-union keys after tuple removals, right? Right. I believe gistVacuumUpdate needed an

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Fri, Jan 20, 2012 at 2:48 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Even if you give the version number in the CREATE EXTENSION command, it's by convention that people actually maintain a sane versioning policy. If people don't take version management seriously, you

Re: [HACKERS] gistVacuumUpdate

2012-01-20 Thread Jaime Casanova
On Wed, Jan 18, 2012 at 10:05 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 13.01.2012 06:24, YAMAMOTO Takashi wrote: hi, gistVacuumUpdate was removed when old-style VACUUM FULL was removed. i wonder why. it was not practical and REINDEX is preferred? anyway, the

Re: [HACKERS] Measuring relation free space

2012-01-20 Thread Jaime Casanova
On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: ignoring all non-leaf pages still gives a considerable difference between pgstattuple and relation_free_space() pgstattuple() counts the single B-tree meta

[HACKERS] REVIEW: pg_stat_statements with query tree based normalization

2012-01-20 Thread Kääriäinen Anssi
This is a short review of pg_stat_statements based on quick testing of the feature. 1. Installation: after managing to actually build PostgreSQL and contrib modules + changing shared_preload_libraries to include pg_stat_statements I got this error: FATAL: could not create shared memory

Re: [HACKERS] Measuring relation free space

2012-01-20 Thread Noah Misch
On Fri, Jan 20, 2012 at 07:03:22PM -0500, Jaime Casanova wrote: On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: ignoring all non-leaf pages still gives a considerable difference between pgstattuple and

Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?

2012-01-20 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:02 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Jan 18, 2012 at 11:12 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jan 18, 2012 at 5:49 PM, Simon Riggs si...@2ndquadrant.com wrote: Can I just check with you that the only review comment is a one line

Re: [HACKERS] REVIEW: pg_stat_statements with query tree based normalization

2012-01-20 Thread Peter Geoghegan
On 21 January 2012 00:24, Kääriäinen Anssi anssi.kaariai...@thl.fi wrote: I did not see any noticeable difference in runtimes with pg_stat_statements installed or uninstalled (as extension). Not tested on master without the patch at all. Overall the feature seems to be really useful. Thanks

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Peter Geoghegan
On 20 January 2012 22:30, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Maybe we should have a heuristic to split a large flush into smaller chunks. The WAL segment boundary would be a quite natural split point, for example, because when crossing the file boundary you have to

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Peter Geoghegan
On 21 January 2012 03:13, Peter Geoghegan pe...@2ndquadrant.com wrote: I have taken the time to re-run the benchmark and update the wiki with that new information - I'd call it a draw. On second though, the occasional latency spikes that we see with my patch (which uses the poll() based latch

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Alexander Korotkov
Hi! Thank you for your feedback! On Fri, Jan 20, 2012 at 3:33 AM, Erik Rijkers e...@xs4all.nl wrote: The patch yields spectacular speedups with small, simple-enough regexen. But it does not do a good enough job when guessing where to use the index and where fall back to Seq Scan. This can

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Alexander Korotkov
On Fri, Jan 20, 2012 at 8:45 PM, Marti Raudsepp ma...@juffo.org wrote: On Fri, Jan 20, 2012 at 01:33, Erik Rijkers e...@xs4all.nl wrote: Btw, it seems impossible to Ctrl-C out of a search once it is submitted; I suppose this is normally necessary for perfomance reasons, but it would be

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Alexander Korotkov
On Fri, Jan 20, 2012 at 12:54 AM, Alexander Korotkov aekorot...@gmail.comwrote: On Fri, Jan 20, 2012 at 12:30 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Apart from that, the multibyte issue seems like the big one. Any way around that? Conversion of pg_wchar to

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Erik Rijkers
On Sat, January 21, 2012 06:26, Alexander Korotkov wrote: Hi! Thank you for your feedback! On Fri, Jan 20, 2012 at 3:33 AM, Erik Rijkers e...@xs4all.nl wrote: The patch yields spectacular speedups with small, simple-enough regexen. But it does not do a good enough job when guessing where

Re: [HACKERS] Finer Extension dependencies

2012-01-20 Thread Hitoshi Harada
On Sun, Dec 18, 2011 at 6:36 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Hi, The extensions work we began in 9.1 is not yet finished entirely (*cough*), so I'm opening a new patch series here by attacking the dependency problems. Some people want us to manage extension version