[HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-01 Thread Fabien COELHO
Add --throttle to pgbench Each client is throttled to the specified rate, which can be expressed in tps or in time (s, ms, us). Throttling is achieved by scheduling transactions along a Poisson-distribution. This is an update of the previous proposal which fix a typo in the sgml

Re: [HACKERS] Graph datatype addition

2013-05-01 Thread Atri Sharma
Hi all, Please find a probable prototype for the same: struct GraphNode { Oid NodeOid;// Oid of the row which is the node here. We will store an identifier to it here rather than the complete row(with data) itself. AdjacencyList *list; // Pointer to the node's adjacency list. };

[HACKERS] fast promotion and log_checkpoints

2013-05-01 Thread Fujii Masao
Hi, In HEAD, when the standby is promoted, recovery requests the checkpoint but doesn't wait for its completion. I found the checkpoint starting log message of this checkpoint looks odd as follows: LOG: checkpoint starting: I think something like the following is better. LOG:

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Simon Riggs
On 30 April 2013 22:54, Jeff Davis pg...@j-davis.com wrote: On Tue, 2013-04-30 at 08:34 -0400, Robert Haas wrote: Uh, wait a minute. I think this is completely wrong. The buffer is LOCKED for this entire sequence of operations. For a checkpoint to happen, it's got to write every buffer,

Re: [HACKERS] Remaining beta blockers

2013-05-01 Thread Simon Riggs
On 27 April 2013 15:59, Tom Lane t...@sss.pgh.pa.us wrote: 2. The checksum algorithm business. Again, we don't get to tinker with that anymore once we're in beta. Checksum changes to output value and control file are now complete and we are ready to go to beta with it. Robert has an

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

2013-05-01 Thread Fujii Masao
On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs si...@2ndquadrant.com wrote: Make fast promotion the default promotion mode. Continue to allow a request for synchronous checkpoints as a mechanism in case of problems. Is there clean way to request synchronous checkpoint at the standby promotion?

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

2013-05-01 Thread Simon Riggs
On 1 May 2013 11:25, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs si...@2ndquadrant.com wrote: Make fast promotion the default promotion mode. Continue to allow a request for synchronous checkpoints as a mechanism in case of problems. Is there clean

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

2013-05-01 Thread Fujii Masao
On Wed, May 1, 2013 at 9:20 PM, Simon Riggs si...@2ndquadrant.com wrote: On 1 May 2013 11:25, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs si...@2ndquadrant.com wrote: Make fast promotion the default promotion mode. Continue to allow a request for

[HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread David Fetter
On Tue, Apr 30, 2013 at 07:53:27PM -0400, Tom Lane wrote: adrian.vondendrie...@credativ.de writes: [ recent pg_dump fails against an 8.4 server if old is used as a name ] Yeah. The reason for this is that old was considered a reserved word in 8.4 and before, but since 9.0 it is not

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread Stefan Kaltenbrunner
On 05/01/2013 04:26 PM, David Fetter wrote: On Tue, Apr 30, 2013 at 07:53:27PM -0400, Tom Lane wrote: adrian.vondendrie...@credativ.de writes: [ recent pg_dump fails against an 8.4 server if old is used as a name ] Yeah. The reason for this is that old was considered a reserved word in 8.4

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread Tom Lane
David Fetter da...@fetter.org writes: According to SQL:2003 and SQL:2008 (and the draft standard, if that matters) in section 5.2 of Foundation, both NEW and OLD are reserved words, so we're going to need to re-reserve them to comply. We don't and won't. There are very many other keywords

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Robert Haas
On Tue, Apr 30, 2013 at 5:54 PM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2013-04-30 at 08:34 -0400, Robert Haas wrote: Uh, wait a minute. I think this is completely wrong. The buffer is LOCKED for this entire sequence of operations. For a checkpoint to happen, it's got to write every

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Robert Haas
On Wed, May 1, 2013 at 11:29 AM, Robert Haas robertmh...@gmail.com wrote: I was worried because SyncOneBuffer checks whether it needs writing without taking a content lock, so the exclusive lock doesn't help. That makes sense, because you don't want a checkpoint to have to get a content lock

Re: [HACKERS] Remaining beta blockers

2013-05-01 Thread Robert Haas
On Tue, Apr 30, 2013 at 9:23 PM, Greg Stark st...@mit.edu wrote: Can I ask what is the design goal of unlogged relations? Are they just an optimization so you can load lots of data without generating piles of wal log? Or are they intended to generate zero wal traffic so they can be populated

Re: [HACKERS] materialized view scannability in other DBs

2013-05-01 Thread Robert Haas
On Tue, Apr 30, 2013 at 2:39 PM, Kevin Grittner kgri...@ymail.com wrote: the fact that Oracle has [...] not felt compelled to add a flag of this type, suggests to me that the feature can't be considered mandatory for a minimal implementation. It seems to me pretty fundamental to have a way to

Re: [HACKERS] materialized view scannability in other DBs

2013-05-01 Thread Andres Freund
On 2013-05-01 11:38:36 -0400, Robert Haas wrote: On Tue, Apr 30, 2013 at 2:39 PM, Kevin Grittner kgri...@ymail.com wrote: the fact that Oracle has [...] not felt compelled to add a flag of this type, suggests to me that the feature can't be considered mandatory for a minimal implementation.

Re: [HACKERS] materialized view scannability in other DBs

2013-05-01 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Kevin Grittner kgri...@ymail.com wrote: It seems to me pretty fundamental to have a way to avoid quietly generating completely bogus results I understand that it seems fundamental to you.  What I'm trying to establish is that reasonable people could

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

2013-05-01 Thread Simon Riggs
On 1 May 2013 14:55, Fujii Masao masao.fu...@gmail.com wrote: On Wed, May 1, 2013 at 9:20 PM, Simon Riggs si...@2ndquadrant.com wrote: On 1 May 2013 11:25, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs si...@2ndquadrant.com wrote: Make fast promotion the

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread David Fetter
On Wed, May 01, 2013 at 11:12:28AM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: According to SQL:2003 and SQL:2008 (and the draft standard, if that matters) in section 5.2 of Foundation, both NEW and OLD are reserved words, so we're going to need to re-reserve them to

Re: [HACKERS] [COMMITTERS] pgsql: Add regression test for bug fixed by recent refactoring.

2013-05-01 Thread Tom Lane
I wrote: Of course the select from vv2 should fail as well, but it doesn't, because vv2 is nowhere to be seen in the rangetable passed to the executor. I think the planner is probably trashing the rangetable once it realizes that the query is completely dummy; but this is wrong because we

[HACKERS] Documentation epub format

2013-05-01 Thread Fabien COELHO
Hello devs, I've given a try to the PostgreSQL documentation in epub format. I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks reader, which was clearly not designed for

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Joshua D. Drake
On 05/01/2013 09:27 AM, Fabien COELHO wrote: Hello devs, I've given a try to the PostgreSQL documentation in epub format. I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-01 Thread Greg Smith
On 5/1/13 4:57 AM, Fabien COELHO wrote: The use case of the option is to be able to generate a continuous gentle load for functional tests, eg in a practice session with students or for testing features on a laptop. If you add this to

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Simon Riggs
On 1 May 2013 16:33, Robert Haas robertmh...@gmail.com wrote: On Wed, May 1, 2013 at 11:29 AM, Robert Haas robertmh...@gmail.com wrote: I was worried because SyncOneBuffer checks whether it needs writing without taking a content lock, so the exclusive lock doesn't help. That makes sense,

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread Stefan Kaltenbrunner
On 05/01/2013 06:14 PM, David Fetter wrote: On Wed, May 01, 2013 at 11:12:28AM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: According to SQL:2003 and SQL:2008 (and the draft standard, if that matters) in section 5.2 of Foundation, both NEW and OLD are reserved words, so we're

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread Tom Lane
David Fetter da...@fetter.org writes: On Wed, May 01, 2013 at 11:12:28AM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: According to SQL:2003 and SQL:2008 (and the draft standard, if that matters) in section 5.2 of Foundation, both NEW and OLD are reserved words, so we're going

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: Once upon a time we had multiple books as documentation, then at some point we merged them. It was quite a few years ago. I would agree at this point that we need to consider breaking them up again. The documentation is unwieldy. The reason we

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Andrew Satori
I would second Tom on this, and if ePub is really a longer term goal of the documentation, the various eBook formats have differing levels of support for hyperlinking that would merit retaining everything in a single book that can be linked from direct references. Dru On May 1, 2013, at 1:52

Re: [HACKERS] Documentation epub format

2013-05-01 Thread k...@rice.edu
On Wed, May 01, 2013 at 01:52:43PM -0400, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: Once upon a time we had multiple books as documentation, then at some point we merged them. It was quite a few years ago. I would agree at this point that we need to consider breaking

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Joshua D. Drake
On 05/01/2013 10:52 AM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: Once upon a time we had multiple books as documentation, then at some point we merged them. It was quite a few years ago. I would agree at this point that we need to consider breaking them up again. The

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Joshua D. Drake
On 05/01/2013 10:56 AM, Andrew Satori wrote: I would second Tom on this, and if ePub is really a longer term goal of the documentation, the various eBook formats have differing levels of support for hyperlinking that would merit retaining everything in a single book that can be linked from

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Robert Haas
On Wed, May 1, 2013 at 1:02 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree, but that was in the original coding wasn't it? I believe the problem was introduced by this commit: commit fdf9e21196a6f58c6021c967dc5776a16190f295 Author: Heikki Linnakangas heikki.linnakan...@iki.fi Date: Wed

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Erik Rijkers
On Wed, May 1, 2013 20:13, Joshua D. Drake wrote: I don't think ePub is a problem here, we will have the same problem with PDF. The issue is the sheer size of the manual. If we can solve the cross referencing issue, breaking them up makes sense I would think. I like the one-huge-chunk pdf:

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Andrew Satori
On May 1, 2013, at 2:11 PM, Joshua D. Drake j...@commandprompt.com wrote: On 05/01/2013 10:52 AM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: Once upon a time we had multiple books as documentation, then at some point we merged them. It was quite a few years ago. I

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Ross Reedstrom
On Wed, May 01, 2013 at 09:33:23AM -0700, Joshua D. Drake wrote: On 05/01/2013 09:27 AM, Fabien COELHO wrote: Hello devs, I've given a try to the PostgreSQL documentation in epub format. I must admit that there is a bit of a disappointement as far as the user experience is

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Jeff Davis
On Wed, 2013-05-01 at 11:33 -0400, Robert Haas wrote: The only time the VM and the data page are out of sync during vacuum is after a crash, right? If that's the case, I didn't think it was a big deal to dirty one extra page (should be extremely rare). Am I missing something? The

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Simon Riggs
On 1 May 2013 19:16, Robert Haas robertmh...@gmail.com wrote: On Wed, May 1, 2013 at 1:02 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree, but that was in the original coding wasn't it? I believe the problem was introduced by this commit: commit fdf9e21196a6f58c6021c967dc5776a16190f295

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Josh Berkus
On 05/01/2013 10:52 AM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: Once upon a time we had multiple books as documentation, then at some point we merged them. It was quite a few years ago. I would agree at this point that we need to consider breaking them up again. The

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Josh Berkus
Also the divisions between sections were totally arbitrary and unintuitive. I think it would make a lot more sense to modify the SGML export to create a book per chapter. Also ... why is this discussion not on pgsql-docs, where it belongs? Crossing it over. -- Josh Berkus PostgreSQL

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Jeff Davis
On Wed, 2013-05-01 at 14:16 -0400, Robert Haas wrote: Now that I'm looking at this, I'm a bit confused by the new logic in visibilitymap_set(). When checksums are enabled, we set the page LSN, which is described like this: we need to protect the heap page from being torn. But how does

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Jeff Davis
On Wed, 2013-05-01 at 20:06 +0100, Simon Riggs wrote: Why aren't we writing just one WAL record for this action? ... I thought about that, too. It certainly seems like more than we want to try to do for 9.3 at this point. The other complication is that there's a lot of conditional

Re: [HACKERS] Recovery target 'immediate'

2013-05-01 Thread Bruce Momjian
On Fri, Apr 26, 2013 at 09:48:48AM -0400, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: That said, maybe the easier choice for a *system* (such as v-thingy) would be to simply to the full backup using pg_basebackup -x (or similar), therefor not needing the log archive at all

Re: [HACKERS] Graph datatype addition

2013-05-01 Thread Misa Simic
On Wednesday, May 1, 2013, Atri Sharma wrote: Hi all, Please find a probable prototype for the same: struct GraphNode { Oid NodeOid;// Oid of the row which is the node here. We will store an identifier to it here rather than the complete row(with data) itself. AdjacencyList

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

2013-05-01 Thread Robert Haas
On Wed, May 1, 2013 at 12:10 PM, Simon Riggs si...@2ndquadrant.com wrote: Heikki said ...remove pg_ctl -m fast/smart option altogether. There is no need to expose that to users. So it is no longer exposed to users. If there are others that share that opinion we may change this. /me blinks.

Re: [HACKERS] high io BUT huge amount of free memory

2013-05-01 Thread Bruce Momjian
On Wed, Apr 24, 2013 at 08:39:09AM -0500, Shaun Thomas wrote: On 04/24/2013 08:24 AM, Robert Haas wrote: Are you referring to the fact that vm.zone_reclaim_mode = 1 is an idiotic default? Well... it is. But even on systems where it's not the default or is explicitly disabled, there's just

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

2013-05-01 Thread Bruce Momjian
On Wed, May 1, 2013 at 07:23:33PM -0400, Robert Haas wrote: On Wed, May 1, 2013 at 12:10 PM, Simon Riggs si...@2ndquadrant.com wrote: Heikki said ...remove pg_ctl -m fast/smart option altogether. There is no need to expose that to users. So it is no longer exposed to users. If there are

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-01 Thread Robert Haas
On Mon, Apr 29, 2013 at 4:33 PM, Jim Nasby j...@nasby.net wrote: On 4/28/13 7:50 AM, Craig Ringer wrote: I find it frustrating that I've never seen an @paraccel email address here and that few of the other vendors of highly customised Pg offshoots are contributing back. It's almost enough to

[HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread Greg Stark
On Wed, May 1, 2013 at 6:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: This is complete nonsense, because David's argument is pretty clearly not nonsense. I think they're valid well reasoned arguments. It's just that the evidence is mixed and on balance leans towards not unnecessarily reserving

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-01 Thread Josh Berkus
I find it frustrating that I've never seen an @paraccel email address here and that few of the other vendors of highly customised Pg offshoots are contributing back. It's almost enough to make me like the GPL. Well, Paraccel never ended up contributing any code, but in years back (2006-2008)

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Greg Stark
On Wed, May 1, 2013 at 5:27 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks reader, which was clearly not designed for handling

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-01 Thread Craig Ringer
On 05/02/2013 08:36 AM, Robert Haas wrote: I've talked with a bunch of other companies over the last few years who also chose PostgreSQL for licensing reasons. I'm pretty happy about that. I think it's a pretty good thing too, personally ... but I do wish they'd contribute a bit more to the

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: ... There are in fact a pretty large number of companies - EnterpriseDB, obviously, but there are many, many others - that are choosing to build businesses around PostgreSQL precisely because it *isn't* GPL. Personally, I think that's a good thing

Re: [HACKERS] Graph datatype addition

2013-05-01 Thread Atri Sharma
Sent from my iPad On 02-May-2013, at 4:33, Misa Simic misa.si...@gmail.com wrote: On Wednesday, May 1, 2013, Atri Sharma wrote: Hi all, Please find a probable prototype for the same: struct GraphNode { Oid NodeOid;// Oid of the row which is the node here. We will store

[HACKERS] Archive Recovery and SR promote command is failed by “contrecord is requested” in ReadRecord()

2013-05-01 Thread KONDO Mitsumasa
Hi, I found that archive recovery and SR promote command is failed by contrecord is requested by 0/420 in ReadRecord(). I investigate about contrecord, it means that record crosses page boundary. I think it is not irregular page, and should be try to read next page in this case. But in

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Peter Eisentraut
On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks reader, which was clearly not designed for handling a 4592 pages

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Gavin Flower
On 02/05/13 15:23, Peter Eisentraut wrote: On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks reader, which was clearly

[HACKERS] \watch stuck on execution of commands returning no tuples

2013-05-01 Thread Michael Paquier
Hi all, When testing \watch, I noticed that process waits indefinitely when executing it with a DDL or a DML. For example: postgres=# CREATE TABLE aa (a int); postgres=# ANALYSE aa \watch 10 -- Process waiting here By referring at do_watch:command.c, the feature is made such as there is no error

Re: [HACKERS] Recovery target 'immediate'

2013-05-01 Thread Michael Paquier
On Thu, May 2, 2013 at 7:40 AM, Bruce Momjian br...@momjian.us wrote: On Fri, Apr 26, 2013 at 09:48:48AM -0400, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: That said, maybe the easier choice for a *system* (such as v-thingy) would be to simply to the full backup using

[HACKERS] Confusing comment in xlog.c or am I missing something?

2013-05-01 Thread Amit Langote
I was just going through the xlog.c and I came across following which confused me: Given, src/include/access/xlogdefs.h #define XLogSegsPerFile (((uint32) 0x) / XLogSegSize) #define XLogFileSize(XLogSegsPerFile * XLogSegSize) Also, typedef struct XLogRecPtr { uint32

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

2013-05-01 Thread Heikki Linnakangas
On 02.05.2013 08:16, Amit Langote wrote: I was just going through the xlog.c and I came across following which confused me: Given, src/include/access/xlogdefs.h #define XLogSegsPerFile (((uint32) 0x) / XLogSegSize) #define XLogFileSize(XLogSegsPerFile * XLogSegSize) Also, typedef

[HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

2013-05-01 Thread Pavan Deolasee
Hi All, While looking at the help message of pg_basebackup, I got a bit confused by it. It looks something like this: Connection options: -d, --dbname=CONNSTR connection string So the long option says --dbname which is not really what its expecting. The same issue also applies to

Re: [HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

2013-05-01 Thread Heikki Linnakangas
On 02.05.2013 08:43, Pavan Deolasee wrote: Hi All, While looking at the help message of pg_basebackup, I got a bit confused by it. It looks something like this: Connection options: -d, --dbname=CONNSTR connection string So the long option says --dbname which is not really what its