Re: [HACKERS] Finer Extension dependencies

2012-01-20 Thread Hitoshi Harada
On Sun, Dec 18, 2011 at 6:36 AM, Dimitri Fontaine wrote: > Hi, > > The extensions work we began in 9.1 is not yet finished entirely > (*cough*), so I'm opening a new patch series here by attacking the > dependency problems. > > Some people want us to manage extension version numbers with sorting >

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Erik Rijkers
On Sat, January 21, 2012 06:26, Alexander Korotkov wrote: > Hi! > > Thank you for your feedback! > > On Fri, Jan 20, 2012 at 3:33 AM, Erik Rijkers wrote: > >> The patch yields spectacular speedups with small, simple-enough regexen. >> But it does not do a >> good enough job when guessing where to

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Alexander Korotkov
On Fri, Jan 20, 2012 at 12:54 AM, Alexander Korotkov wrote: > On Fri, Jan 20, 2012 at 12:30 AM, Heikki Linnakangas < > heikki.linnakan...@enterprisedb.com> wrote: > >> Apart from that, the multibyte issue seems like the big one. Any way >>> around that? >> >> Conversion of pg_wchar to multibyte ch

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Alexander Korotkov
On Fri, Jan 20, 2012 at 8:45 PM, Marti Raudsepp wrote: > On Fri, Jan 20, 2012 at 01:33, Erik Rijkers wrote: > > Btw, it seems impossible to Ctrl-C out of a search once it is submitted; > I suppose this is > > normally necessary for perfomance reasons, but it would be useful te be > able to compi

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Alexander Korotkov
Hi! Thank you for your feedback! On Fri, Jan 20, 2012 at 3:33 AM, Erik Rijkers wrote: > The patch yields spectacular speedups with small, simple-enough regexen. > But it does not do a > good enough job when guessing where to use the index and where fall back > to Seq Scan. This can > lead to

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Peter Geoghegan
On 21 January 2012 03:13, Peter Geoghegan wrote: > I have taken the time to re-run the benchmark and update the wiki with > that new information - I'd call it a draw. On second though, the occasional latency spikes that we see with my patch (which uses the poll() based latch in the run that is be

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Peter Geoghegan
On 20 January 2012 22:30, Heikki Linnakangas wrote: > Maybe we should have a heuristic to split a large flush into smaller chunks. > The WAL segment boundary would be a quite natural split point, for example, > because when crossing the file boundary you have to issue separate fsync()s > for the f

Re: [HACKERS] REVIEW: pg_stat_statements with query tree based normalization

2012-01-20 Thread Peter Geoghegan
On 21 January 2012 00:24, Kääriäinen Anssi wrote: > I did not see any noticeable difference in runtimes with pg_stat_statements > installed or uninstalled (as extension). > Not tested on master without the patch at all. > > Overall the feature seems to be really useful. Thanks for taking the tim

Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?

2012-01-20 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:02 AM, Simon Riggs wrote: > On Wed, Jan 18, 2012 at 11:12 PM, Robert Haas wrote: >> On Wed, Jan 18, 2012 at 5:49 PM, Simon Riggs wrote: >>> Can I just check with you that the only review comment is a one line >>> change? Seems better to make any additional review comme

Re: [HACKERS] Measuring relation free space

2012-01-20 Thread Noah Misch
On Fri, Jan 20, 2012 at 07:03:22PM -0500, Jaime Casanova wrote: > On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch wrote: > > On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: > >> > >> ignoring all non-leaf pages still gives a considerable difference > >> between pgstattuple and relation

[HACKERS] REVIEW: pg_stat_statements with query tree based normalization

2012-01-20 Thread Kääriäinen Anssi
This is a short review of pg_stat_statements based on quick testing of the feature. 1. Installation: after managing to actually build PostgreSQL and contrib modules + changing shared_preload_libraries to include pg_stat_statements I got this error: FATAL: could not create shared memory segment:

Re: [HACKERS] Measuring relation free space

2012-01-20 Thread Jaime Casanova
On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch wrote: > On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: >> >> ignoring all non-leaf pages still gives a considerable difference >> between pgstattuple and relation_free_space() > > pgstattuple() counts the single B-tree meta page as alwa

Re: [HACKERS] gistVacuumUpdate

2012-01-20 Thread Jaime Casanova
On Wed, Jan 18, 2012 at 10:05 AM, Heikki Linnakangas wrote: > On 13.01.2012 06:24, YAMAMOTO Takashi wrote: >> >> hi, >> >> gistVacuumUpdate was removed when old-style VACUUM FULL was removed. >> i wonder why. >> it was not practical and REINDEX is preferred? >> >> anyway, the removal seems incompl

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Fri, Jan 20, 2012 at 2:48 PM, Heikki Linnakangas wrote: > Even if you give the version number in the CREATE EXTENSION command, it's by > convention that people actually maintain a sane versioning policy. If people > don't take version management seriously, you will quickly end up with five > di

Re: [HACKERS] gistVacuumUpdate

2012-01-20 Thread Heikki Linnakangas
On 18.01.2012 23:38, YAMAMOTO Takashi wrote: i'm wondering because what gistVacuumUpdate used to do does not seem to be necessarily tied to the old-style VACUUM FULL. currently, no one will re-union keys after tuple removals, right? Right. I believe gistVacuumUpdate needed an AccessExclusiveLoc

Re: [HACKERS] Inline Extension

2012-01-20 Thread Heikki Linnakangas
On 21.01.2012 00:00, Daniel Farina wrote: I think this is somewhat rube-goldberg-esque, and denies non-superuser roles the ability to get more version management of schema and operators. As-is many organizations are submitting "migrations" via plain SQL that include committing to a version manag

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Heikki Linnakangas
I spent some time cleaning this up. Details below, but here are the highlights: * Reverted the removal of wal_writer_delay * Doesn't rely on WAL writer. Any process can act as the "leader" now. * Removed heuristic on big flushes * Uses PGSemaphoreLock/Unlock instead of latches On 20.01.2012 17:

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Thu, Jan 19, 2012 at 8:15 AM, Tom Lane wrote: >> Heikki Linnakangas writes: >>> Frankly I don't see the point of this. If the extension is an independent >>> piece of (SQL) code, developed separately from an application, with its own >>> lifecycle, a .sql file seems like the best way to distri

Re: [HACKERS] Remembering bug #6123

2012-01-20 Thread Kevin Grittner
Tom Lane wrote: > I'm up to my elbows in planner guts at the moment, but will try to > fix up the patch this weekend if you want. They have scheduled testers to check on this issue next week, so it would be great to get as close as we can on the stuff that matters. -Kevin -- Sent via pgsql

Re: [HACKERS] Remembering bug #6123

2012-01-20 Thread Tom Lane
"Kevin Grittner" writes: > After a couple meetings, I have approval to get this into an > application release currently in development. Assuming that your > patch from the 13th is good for doing the testing, I think I can > post test results in about three weeks. I'll also work on a > follow-on

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 2:21 PM, Tom Lane wrote: > Robert Haas writes: >> Err, actually, now that I think about it, that might be a problem: >> what happens if we're trying to test two characters for equality and >> the encoding conversion fails? > > This is surely all entirely doable given the e

Re: [HACKERS] Checkpoint sync pause

2012-01-20 Thread Robert Haas
On Mon, Jan 16, 2012 at 8:59 PM, Greg Smith wrote: > [ interesting description of problem scenario and necessary conditions for > reproducing it ] This is about what I thought was happening, but I'm still not quite sure how to recreate it in the lab. Have you had a chance to test with Linux 3.2

Re: [HACKERS] Command Triggers

2012-01-20 Thread Dimitri Fontaine
Tom Lane writes: > For the creation case, RangeVarGetCreationNamespace should handle that. > The code Dimitri quoted is wrong, but not that hard to fix. Ok. > Unfortunately, the code he quoted for the ALTER case is also wrong, > and harder to fix. Until you've done the lookup you don't know whi

Re: [HACKERS] Command Triggers

2012-01-20 Thread Tom Lane
Robert Haas writes: > Hmm, OK. But what happens if the user doesn't specify a schema name > explicitly? For the creation case, RangeVarGetCreationNamespace should handle that. The code Dimitri quoted is wrong, but not that hard to fix. Unfortunately, the code he quoted for the ALTER case is al

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Tom Lane
Robert Haas writes: > Err, actually, now that I think about it, that might be a problem: > what happens if we're trying to test two characters for equality and > the encoding conversion fails? This is surely all entirely doable given the encoding infrastructure we already have. We might need som

Re: [HACKERS] Inline Extension

2012-01-20 Thread Dimitri Fontaine
Robert Haas writes: >> Maybe you would want the system to be able to determine the oldest >> version to start from to reach the current default_version given in the >> control file, but I guess it would be better to add another property >> like default_full_version or such (last_stop?). > > Possib

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:35 PM, Greg Smith wrote: > On 01/20/2012 10:37 AM, Robert Haas wrote: >> >> On Thu, Jan 19, 2012 at 11:29 PM, Greg Smith  wrote: >>> >>> vacuum_cost_page_hit = 0.1 >>> >>> vacuum_cost_page_miss = 1.0 >>> vacuum_cost_page_dirty = 2.0 >>> >>> Now add in the new setting, wh

Re: [HACKERS] Command Triggers

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:14 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> I think the OID is better than the name, but if it's easy to pass the >> name and schema, then I'm fine with it.  But I do think this is one of > > It's quite easy to get name and schema from the command yes, here'

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:14 PM, David E. Wheeler wrote: > On Jan 20, 2012, at 8:58 AM, Robert Haas wrote: > >> If, however, >> we're not using UTF-8, we have to first turn \u into a Unicode >> code point, then covert that to a character in the database encoding, >> and then test for equality

Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-20 Thread Heikki Linnakangas
On 04.01.2012 13:14, Simon Riggs wrote: On Tue, Jan 3, 2012 at 11:28 PM, Tom Lane wrote: Jim Nasby writes: On Jan 3, 2012, at 12:11 PM, Simon Riggs wrote: This could well be related to the fact that DropRelFileNodeBuffers() does a scan of shared_buffers, which is an O(N) approach no matter t

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-20 Thread Greg Smith
On 01/20/2012 10:37 AM, Robert Haas wrote: On Thu, Jan 19, 2012 at 11:29 PM, Greg Smith wrote: vacuum_cost_page_hit = 0.1 vacuum_cost_page_miss = 1.0 vacuum_cost_page_dirty = 2.0 Now add in the new setting, which is explicitly said to be the read value: vacuum_cost_read_limit = 8000 # maximum

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Andrew Dunstan
On 01/20/2012 11:58 AM, Robert Haas wrote: On Fri, Jan 20, 2012 at 10:45 AM, Andrew Dunstan wrote: XML's&#; escape mechanism is more or less the equivalent of JSON's \u. But XML documents can be encoded in a variety of encodings, including non-unicode encodings such as Latin-1. Howeve

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Garick Hamlin
On Fri, Jan 20, 2012 at 09:12:13AM -0800, David E. Wheeler wrote: > On Jan 19, 2012, at 9:07 PM, Tom Lane wrote: > > > If his client encoding is UTF8, the value will be letter-perfect JSON > > when it gets to him; and if his client encoding is not UTF8, then he's > > already pretty much decided th

Re: [HACKERS] Remembering bug #6123

2012-01-20 Thread Kevin Grittner
"Kevin Grittner" wrote: > Tom Lane wrote: >> Well, the bottom line that's concerning me here is whether >> throwing errors is going to push anyone's application into an >> unfixable corner. I'm somewhat encouraged that your Circuit >> Courts software can adapt to it, since that's certainly one

Re: [HACKERS] Command Triggers

2012-01-20 Thread Dimitri Fontaine
Robert Haas writes: > I think the OID is better than the name, but if it's easy to pass the > name and schema, then I'm fine with it. But I do think this is one of It's quite easy to get name and schema from the command yes, here's an example of how I'm doing it for some commands:

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread David E. Wheeler
On Jan 20, 2012, at 8:58 AM, Robert Haas wrote: > If, however, > we're not using UTF-8, we have to first turn \u into a Unicode > code point, then covert that to a character in the database encoding, > and then test for equality with the other character after that. I'm > not sure whether that

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread David E. Wheeler
On Jan 19, 2012, at 9:07 PM, Tom Lane wrote: > If his client encoding is UTF8, the value will be letter-perfect JSON > when it gets to him; and if his client encoding is not UTF8, then he's > already pretty much decided that he doesn't give a fig about the > Unicode-centricity of the JSON spec, no

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:45 AM, Andrew Dunstan wrote: > XML's &#; escape mechanism is more or less the equivalent of JSON's > \u. But XML documents can be encoded in a variety of encodings, > including non-unicode encodings such as Latin-1. However, no matter what the > document encoding

Re: [HACKERS] Inline Extension

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 11:29 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> peeves is that the system doesn't know how to do an install of v1.1 by >> running the v1.0 script followed by the 1.0-1.1 upgrade script, which > > Did you try > >  CREATE EXTENSION foo FROM 1.0; Well, yes, that w

Re: [HACKERS] WIP: index support for regexp search

2012-01-20 Thread Marti Raudsepp
On Fri, Jan 20, 2012 at 01:33, Erik Rijkers wrote: > Btw, it seems impossible to Ctrl-C out of a search once it is submitted; I > suppose this is > normally necessary for perfomance reasons, but it would be useful te be able > to compile a test > version that allows it. I believe being interrup

Re: [HACKERS] Inline Extension

2012-01-20 Thread Dimitri Fontaine
Robert Haas writes: > peeves is that the system doesn't know how to do an install of v1.1 by > running the v1.0 script followed by the 1.0-1.1 upgrade script, which Did you try CREATE EXTENSION foo FROM 1.0; Maybe you would want the system to be able to determine the oldest version to start f

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: > On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >> I've taken that idea and used it to build a second Clog cache, known >> as ClogHistory which allows access to the read-only tail of pages in >> the clog. Once a page has been written to for

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Andrew Dunstan
On 01/20/2012 09:19 AM, Robert Haas wrote: On Fri, Jan 20, 2012 at 12:07 AM, Tom Lane wrote: Andrew Dunstan writes: On 01/19/2012 04:12 PM, Robert Haas wrote: On Thu, Jan 19, 2012 at 4:07 PM, Andrew Dunstanwrote: The spec only allows unescaped Unicode chars (and for our purposes that

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:38 AM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 3:32 PM, Robert Haas wrote: >> On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs wrote: >>> On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: > I've taken th

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 3:32 PM, Robert Haas wrote: > On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs wrote: >> On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: >>> On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: I've taken that idea and used it to build a second Clog cache, known

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-20 Thread Robert Haas
On Thu, Jan 19, 2012 at 11:29 PM, Greg Smith wrote: > I chewed a bit on Heikki's comment that similarity to the query planning > parameters might be useful, and Robert's that being able to explain how the > feature works more easily has value.  I have an initial adjustment of my > general idea tha

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: >> On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >>> I've taken that idea and used it to build a second Clog cache, known >>> as ClogHistory which allows access to the read-only tail o

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:27 AM, Tom Lane wrote: > Robert Haas writes: >> The code I've written so far does no canonicalization of the input >> value of any kind, just as we do for XML. > > Fair enough. > >> So, given that framework, what the patch does is this: if you're using >> UTF-8, then \u

Re: [HACKERS] Group commit, revised

2012-01-20 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:46 PM, Peter Geoghegan wrote: > On 19 January 2012 17:40, Robert Haas wrote: >> I don't know what you mean by this.  I think removing wal_writer_delay >> is premature, because I think it still may have some utility, and the >> patch removes it.  That's a separate change

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Tom Lane
Robert Haas writes: > The code I've written so far does no canonicalization of the input > value of any kind, just as we do for XML. Fair enough. > So, given that framework, what the patch does is this: if you're using > UTF-8, then \u is accepted, provided that is something that > equa

Re: [HACKERS] Patch review for logging hooks (CF 2012-01)

2012-01-20 Thread Marti Raudsepp
On Fri, Jan 20, 2012 at 17:01, Robert Haas wrote: > We should probably at least have a > working example for testing purposes, though, whether or not we end up > committing it. Martin Pihlak sent a short description of how to test the patch with his pg_logforward module: http://archives.postgresq

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 9:44 AM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: >> On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >>> I've taken that idea and used it to build a second Clog cache, known >>> as ClogHistory which allows access to the read-only tail of

Re: [HACKERS] Command Triggers

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 9:28 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> My advice is to forget about trying to provide the command string to >> the user for the first version of this patch.  As you're finding out, >> there's no simple, easy, obvious way of doing it, and there are N>0 >>

Re: [HACKERS] Patch review for logging hooks (CF 2012-01)

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 2:47 AM, Greg Smith wrote: >> The updated patch looks good, marking as 'Ready for Committer' > > Patches without documentation are never ready for commit.  For this one, I'm > not sure if that should be in the form of a reference example in contrib, or > just something that

Re: [HACKERS] our buffer replacement strategy is kind of lame

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 9:29 AM, Heikki Linnakangas wrote: > I'd like to see some benchmarks that show a benefit from these patches, > before committing something like this that complicates the code. These > patches are fairly small, but nevertheless. Once we have a test case, we can > argue wheth

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: > On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >> I've taken that idea and used it to build a second Clog cache, known >> as ClogHistory which allows access to the read-only tail of pages in >> the clog. Once a page has been written to for

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-20 Thread Marko Kreen
On Tue, Jan 17, 2012 at 05:53:33PM +0900, Kyotaro HORIGUCHI wrote: > Hello, This is revised and rebased version of the patch. > > a. Old term `Add Tuple Function' is changed to 'Store >Handler'. The reason why not `storage' is simply length of the >symbols. > > b. I couldn't find the pla

Re: [HACKERS] Inline Extension

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 8:52 AM, Dimitri Fontaine wrote: > Tom Lane writes: >> Robert Haas writes: >>> I guess the question is: for what purpose? >> >> Indeed, it seems like such a thing is not an extension at all anymore, >> or at least it gives up many of the useful properties of extensions. >

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 2:11 PM, Heikki Linnakangas wrote: > On 20.01.2012 15:32, Robert Haas wrote: >> >> On Sat, Jan 14, 2012 at 9:32 AM, Heikki Linnakangas >>  wrote: >>> >>> Here's another version of the patch to make XLogInsert less of a >>> bottleneck >>> on multi-CPU systems. The basic ide

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 12:54 PM, Fujii Masao wrote: > Thanks for the review! > > On Fri, Jan 20, 2012 at 8:15 PM, Simon Riggs wrote: >> I'm looking at this patch and wondering why we're doing so many >> press-ups to ensure full_page_writes parameter is on. This will still >> fail if you use a ut

Re: [HACKERS] pg_basebackup option for handling symlinks

2012-01-20 Thread Magnus Hagander
On Mon, Jan 16, 2012 at 19:52, Peter Eisentraut wrote: > On sön, 2012-01-08 at 22:22 +0100, Magnus Hagander wrote: >> On Sun, Jan 8, 2012 at 21:53, Peter Eisentraut wrote: >> > I've recently had a possible need for telling pg_basebackup how to >> > handle symlinks in the remote data directory, in

Re: [HACKERS] our buffer replacement strategy is kind of lame

2012-01-20 Thread Heikki Linnakangas
On 03.01.2012 17:56, Simon Riggs wrote: On Tue, Jan 3, 2012 at 3:18 PM, Robert Haas wrote: 2. When a backend can't find a free buffer, it spins for a long time while holding the lock. This makes the buffer strategy O(N) in its worst case, which slows everything down. Notably, while this is hap

Re: [HACKERS] Command Triggers

2012-01-20 Thread Dimitri Fontaine
Robert Haas writes: > My advice is to forget about trying to provide the command string to > the user for the first version of this patch. As you're finding out, > there's no simple, easy, obvious way of doing it, and there are N>0 > useful things that can be done without that functionality. Act

Re: [HACKERS] JSON for PG 9.2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 12:07 AM, Tom Lane wrote: > Andrew Dunstan writes: >> On 01/19/2012 04:12 PM, Robert Haas wrote: >>> On Thu, Jan 19, 2012 at 4:07 PM, Andrew Dunstan  wrote: The spec only allows unescaped Unicode chars (and for our purposes that means UTF8). An unescaped non-ASCI

Re: [HACKERS] Inline Extension

2012-01-20 Thread Dimitri Fontaine
Tom Lane writes: > Robert Haas writes: >> I guess the question is: for what purpose? > > Indeed, it seems like such a thing is not an extension at all anymore, > or at least it gives up many of the useful properties of extensions. I'm thinking that a common name and version number tracked in the

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: > I've taken that idea and used it to build a second Clog cache, known > as ClogHistory which allows access to the read-only tail of pages in > the clog. Once a page has been written to for the last time, it will > be accessed via the ClogHistory

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-01-20 Thread Robert Haas
On Sat, Jan 14, 2012 at 9:32 AM, Heikki Linnakangas wrote: > Here's another version of the patch to make XLogInsert less of a bottleneck > on multi-CPU systems. The basic idea is the same as before, but several bugs > have been fixed, and lots of misc. clean up has been done. This seems to need a

Re: [HACKERS] pg_basebackup is not checking IDENTIFY_SYSTEM numbre of columns

2012-01-20 Thread Magnus Hagander
On Sun, Jan 15, 2012 at 22:00, Jaime Casanova wrote: > On Wed, Jan 11, 2012 at 5:11 PM, Magnus Hagander wrote: >> >> No, no reason. Adding such a check would be a good idea. >> > > ok. patch attached, it also adds a few PQclear() calls before > disconnect_and_exit(). I don't think we need to car

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Fujii Masao
Thanks for the review! On Fri, Jan 20, 2012 at 8:15 PM, Simon Riggs wrote: > I'm looking at this patch and wondering why we're doing so many > press-ups to ensure full_page_writes parameter is on. This will still > fail if you use a utility that removes the full page writes, but fail > silently.

Re: [HACKERS] PATCH: tracking temp files in pg_stat_database

2012-01-20 Thread Magnus Hagander
On Tue, Jan 17, 2012 at 21:39, Tomas Vondra wrote: > On 20.12.2011 19:59, Tomas Vondra wrote: >> On 20.12.2011 11:20, Magnus Hagander wrote: >>> 2011/12/20 Tomas Vondra : I haven't updated the docs yet - let's see if the patch is acceptable at all first. >>> >>> Again, without havin

Re: [HACKERS] New replication mode: write

2012-01-20 Thread Simon Riggs
On Mon, Jan 16, 2012 at 4:17 PM, Simon Riggs wrote: > On Mon, Jan 16, 2012 at 12:45 PM, Fujii Masao wrote: > >> Done. Attached is the updated version of the patch. > > Thanks. > > I'll review this first, but can't start immediately. Please expect > something back in 2 days. On initial review thi

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 11:04 AM, Fujii Masao wrote: > But Steve encountered it again, which means that the above fix is not > sufficient. Unless the issue is derived from my patch, we should do > another cycle of diagnosis of it. It's my bug, and I've posted a fix but not yet applied it, just a

Re: [HACKERS] IDLE in transaction introspection

2012-01-20 Thread Magnus Hagander
On Thu, Jan 19, 2012 at 15:42, Fujii Masao wrote: > On Thu, Jan 19, 2012 at 10:31 PM, Magnus Hagander wrote: >> Applied with fairly extensive modifications. I moved things around, >> switched to using enum instead of int+#define and a few things like >> that. Also changed most of the markup in th

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Simon Riggs
On Tue, Jan 17, 2012 at 10:38 AM, Fujii Masao wrote: > On Fri, Jan 13, 2012 at 5:02 PM, Fujii Masao wrote: >> The amount of code changes to allow pg_basebackup to make a backup from >> the standby seems to be small. So I ended up merging that changes and the >> infrastructure patch. WIP patch att

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Fujii Masao
On Fri, Jan 20, 2012 at 7:37 PM, Erik Rijkers wrote: > I'm not sure, but it does look like this is the "mystery" bug that I > encountered repeatedly > already in 9.0devel; but I was never able to reproduce it reliably.  But I > don't think it was ever > solved. > >  http://archives.postgresql.or

Re: [HACKERS] WAL Restore process during recovery

2012-01-20 Thread Fujii Masao
On Fri, Jan 20, 2012 at 7:38 PM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 3:43 AM, Fujii Masao wrote: > > Requested update Thanks! Will review. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] WAL Restore process during recovery

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 3:43 AM, Fujii Masao wrote: Requested update --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Support, Training & Services diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index ce659ec..469e6d

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Erik Rijkers
On Fri, January 20, 2012 05:01, Steve Singer wrote: > On 12-01-17 05:38 AM, Fujii Masao wrote: >> On Fri, Jan 13, 2012 at 5:02 PM, Fujii Masao wrote: >>> The amount of code changes to allow pg_basebackup to make a backup from >>> the standby seems to be small. So I ended up merging that changes an

Re: [HACKERS] Online base backup from the hot-standby

2012-01-20 Thread Fujii Masao
On Fri, Jan 20, 2012 at 1:01 PM, Steve Singer wrote: > Here is my review of this verison of the patch. I think this patch has been > in every CF for 9.2 and I feel it is getting close to being committed. Thanks for the review! > Testing Review > > > I encountered