[HACKERS] [PATCH] add --throttle to pgbench (submission 3)

Add --throttle to pgbench Each client is throttled to the specified rate, which can be expressed in tps or in time (s, ms, us). Throttling is achieved by scheduling transactions along a Poisson-distribution. This is an update of the previous proposal which fix a typo in the sgml documentati

Re: [HACKERS] Graph datatype addition

Hi all, Please find a probable prototype for the same: struct GraphNode { Oid NodeOid;// Oid of the row which is the node here. We will store an identifier to it here rather than the complete row(with data) itself. AdjacencyList *list; // Pointer to the node's adjacency list. }; st

[HACKERS] fast promotion and log_checkpoints

Hi, In HEAD, when the standby is promoted, recovery requests the checkpoint but doesn't wait for its completion. I found the checkpoint starting log message of this checkpoint looks odd as follows: LOG: checkpoint starting: I think something like the following is better. LOG: checkpoi

Re: [HACKERS] corrupt pages detected by enabling checksums

On 30 April 2013 22:54, Jeff Davis wrote: > On Tue, 2013-04-30 at 08:34 -0400, Robert Haas wrote: >> Uh, wait a minute. I think this is completely wrong. The buffer is >> LOCKED for this entire sequence of operations. For a checkpoint to >> "happen", it's got to write every buffer, which it wil

Re: [HACKERS] Remaining beta blockers

On 27 April 2013 15:59, Tom Lane wrote: > 2. The checksum algorithm business. Again, we don't get to tinker with > that anymore once we're in beta. Checksum changes to output value and control file are now complete and we are ready to go to beta with it. Robert has an outstanding comment on vi

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs wrote: > Make fast promotion the default promotion mode. > Continue to allow a request for synchronous > checkpoints as a mechanism in case of problems. Is there clean way to request synchronous checkpoint at the standby promotion? I'm sure that we can

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

On 1 May 2013 11:25, Fujii Masao wrote: > On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs wrote: >> Make fast promotion the default promotion mode. >> Continue to allow a request for synchronous >> checkpoints as a mechanism in case of problems. > > Is there clean way to request synchronous checkpoi

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

On Wed, May 1, 2013 at 9:20 PM, Simon Riggs wrote: > On 1 May 2013 11:25, Fujii Masao wrote: >> On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs wrote: >>> Make fast promotion the default promotion mode. >>> Continue to allow a request for synchronous >>> checkpoints as a mechanism in case of proble

[HACKERS] Re: [BUGS] BUG #8128: pg_dump (>= 9.1) failed while dumping a scheme named "old" from PostgreSQL 8.4

On Tue, Apr 30, 2013 at 07:53:27PM -0400, Tom Lane wrote: > adrian.vondendrie...@credativ.de writes: > > [ recent pg_dump fails against an 8.4 server if "old" is used as a > > name ] > > Yeah. The reason for this is that "old" was considered a reserved > word in 8.4 and before, but since 9.0 it i

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (>= 9.1) failed while dumping a scheme named "old" from PostgreSQL 8.4

On 05/01/2013 04:26 PM, David Fetter wrote: > On Tue, Apr 30, 2013 at 07:53:27PM -0400, Tom Lane wrote: >> adrian.vondendrie...@credativ.de writes: >>> [ recent pg_dump fails against an 8.4 server if "old" is used as a >>> name ] >> >> Yeah. The reason for this is that "old" was considered a reser

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (>= 9.1) failed while dumping a scheme named "old" from PostgreSQL 8.4

David Fetter writes: > According to SQL:2003 and SQL:2008 (and the draft standard, if that > matters) in section 5.2 of Foundation, both NEW and OLD are reserved > words, so we're going to need to re-reserve them to comply. We don't and won't. There are very many other keywords that are less res

Re: [HACKERS] corrupt pages detected by enabling checksums

On Tue, Apr 30, 2013 at 5:54 PM, Jeff Davis wrote: > On Tue, 2013-04-30 at 08:34 -0400, Robert Haas wrote: >> Uh, wait a minute. I think this is completely wrong. The buffer is >> LOCKED for this entire sequence of operations. For a checkpoint to >> "happen", it's got to write every buffer, whi

Re: [HACKERS] corrupt pages detected by enabling checksums

On Wed, May 1, 2013 at 11:29 AM, Robert Haas wrote: >> I was worried because SyncOneBuffer checks whether it needs writing >> without taking a content lock, so the exclusive lock doesn't help. That >> makes sense, because you don't want a checkpoint to have to get a >> content lock on every buffer

Re: [HACKERS] Remaining beta blockers

On Tue, Apr 30, 2013 at 9:23 PM, Greg Stark wrote: > Can I ask what is the design goal of unlogged relations? Are they just > an optimization so you can load lots of data without generating piles > of wal log? Or are they intended to generate zero wal traffic so they > can be populated on a standb

Re: [HACKERS] materialized view scannability in other DBs

On Tue, Apr 30, 2013 at 2:39 PM, Kevin Grittner wrote: >> the fact that Oracle has [...] not felt compelled to add a flag >> of this type, suggests to me that the feature can't be considered >> mandatory for a minimal implementation. > > It seems to me pretty fundamental to have a way to avoid qui

Re: [HACKERS] materialized view scannability in other DBs

On 2013-05-01 11:38:36 -0400, Robert Haas wrote: > On Tue, Apr 30, 2013 at 2:39 PM, Kevin Grittner wrote: > >> the fact that Oracle has [...] not felt compelled to add a flag > >> of this type, suggests to me that the feature can't be considered > >> mandatory for a minimal implementation. > > > >

Re: [HACKERS] materialized view scannability in other DBs

Robert Haas wrote: > Kevin Grittner wrote: >> It seems to me pretty fundamental to have a way to avoid quietly >> generating completely bogus results > I understand that it seems fundamental to you.  What I'm trying > to establish is that reasonable people could disagree about that. > I think t

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

On 1 May 2013 14:55, Fujii Masao wrote: > On Wed, May 1, 2013 at 9:20 PM, Simon Riggs wrote: >> On 1 May 2013 11:25, Fujii Masao wrote: >>> On Wed, Apr 24, 2013 at 8:49 PM, Simon Riggs wrote: Make fast promotion the default promotion mode. Continue to allow a request for synchronous >

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (>= 9.1) failed while dumping a scheme named "old" from PostgreSQL 8.4

On Wed, May 01, 2013 at 11:12:28AM -0400, Tom Lane wrote: > David Fetter writes: > > According to SQL:2003 and SQL:2008 (and the draft standard, if > > that matters) in section 5.2 of Foundation, both NEW and OLD are > > reserved words, so we're going to need to re-reserve them to > > comply. > >

Re: [HACKERS] [COMMITTERS] pgsql: Add regression test for bug fixed by recent refactoring.

I wrote: > Of course the select from vv2 should fail as well, but it doesn't, > because vv2 is nowhere to be seen in the rangetable passed to the > executor. I think the planner is probably trashing the rangetable > once it realizes that the query is completely dummy; but this is wrong > because w

[HACKERS] Documentation epub format

Hello devs, I've given a try to the PostgreSQL documentation in epub format. I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks reader, which was clearly not designed for ha

Re: [HACKERS] Documentation epub format

On 05/01/2013 09:27 AM, Fabien COELHO wrote: Hello devs, I've given a try to the PostgreSQL documentation in epub format. I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

On 5/1/13 4:57 AM, Fabien COELHO wrote: The use case of the option is to be able to generate a continuous gentle load for functional tests, eg in a practice session with students or for testing features on a laptop. If you add this to https://commitfest.postgresql.org/action/commitfest_view?id

Re: [HACKERS] corrupt pages detected by enabling checksums

On 1 May 2013 16:33, Robert Haas wrote: > On Wed, May 1, 2013 at 11:29 AM, Robert Haas wrote: >>> I was worried because SyncOneBuffer checks whether it needs writing >>> without taking a content lock, so the exclusive lock doesn't help. That >>> makes sense, because you don't want a checkpoint to

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (>= 9.1) failed while dumping a scheme named "old" from PostgreSQL 8.4

On 05/01/2013 06:14 PM, David Fetter wrote: > On Wed, May 01, 2013 at 11:12:28AM -0400, Tom Lane wrote: >> David Fetter writes: >>> According to SQL:2003 and SQL:2008 (and the draft standard, if >>> that matters) in section 5.2 of Foundation, both NEW and OLD are >>> reserved words, so we're going

Re: [HACKERS] Re: [BUGS] BUG #8128: pg_dump (>= 9.1) failed while dumping a scheme named "old" from PostgreSQL 8.4

David Fetter writes: > On Wed, May 01, 2013 at 11:12:28AM -0400, Tom Lane wrote: >> David Fetter writes: >>> According to SQL:2003 and SQL:2008 (and the draft standard, if >>> that matters) in section 5.2 of Foundation, both NEW and OLD are >>> reserved words, so we're going to need to re-reserve

Re: [HACKERS] Documentation epub format

"Joshua D. Drake" writes: > Once upon a time we had multiple books as documentation, then at some > point we merged them. It was quite a few years ago. > I would agree at this point that we need to consider breaking them up > again. The documentation is unwieldy. The reason we merged them was t

Re: [HACKERS] Documentation epub format

I would second Tom on this, and if ePub is really a longer term goal of the documentation, the various eBook formats have differing levels of support for hyperlinking that would merit retaining everything in a single book that can be linked from direct references. Dru On May 1, 2013, at 1:52 P

Re: [HACKERS] Documentation epub format

On Wed, May 01, 2013 at 01:52:43PM -0400, Tom Lane wrote: > "Joshua D. Drake" writes: > > Once upon a time we had multiple books as documentation, then at some > > point we merged them. It was quite a few years ago. > > I would agree at this point that we need to consider breaking them up > > ag

Re: [HACKERS] Documentation epub format

On 05/01/2013 10:52 AM, Tom Lane wrote: "Joshua D. Drake" writes: Once upon a time we had multiple books as documentation, then at some point we merged them. It was quite a few years ago. I would agree at this point that we need to consider breaking them up again. The documentation is unwieldy

Re: [HACKERS] Documentation epub format

On 05/01/2013 10:56 AM, Andrew Satori wrote: I would second Tom on this, and if ePub is really a longer term goal of the documentation, the various eBook formats have differing levels of support for hyperlinking that would merit retaining everything in a single book that can be linked from d

Re: [HACKERS] corrupt pages detected by enabling checksums

On Wed, May 1, 2013 at 1:02 PM, Simon Riggs wrote: > I agree, but that was in the original coding wasn't it? I believe the problem was introduced by this commit: commit fdf9e21196a6f58c6021c967dc5776a16190f295 Author: Heikki Linnakangas Date: Wed Feb 13 17:46:23 2013 +0200 Update visibil

Re: [HACKERS] Documentation epub format

On Wed, May 1, 2013 20:13, Joshua D. Drake wrote: > I don't think ePub is a problem here, we will have the same problem with > PDF. The issue is the sheer size of the manual. If we can solve the > cross referencing issue, breaking them up makes sense I would think. I like the one-huge-chunk pdf:

Re: [HACKERS] Documentation epub format

On May 1, 2013, at 2:11 PM, "Joshua D. Drake" wrote: > > On 05/01/2013 10:52 AM, Tom Lane wrote: >> "Joshua D. Drake" writes: >>> Once upon a time we had multiple books as documentation, then at some >>> point we merged them. It was quite a few years ago. >>> I would agree at this point that w

Re: [HACKERS] Documentation epub format

On Wed, May 01, 2013 at 09:33:23AM -0700, Joshua D. Drake wrote: > > On 05/01/2013 09:27 AM, Fabien COELHO wrote: > > > > > >Hello devs, > > > >I've given a try to the PostgreSQL documentation in epub format. > > > >I must admit that there is a bit of a disappointement as far as the user > >experi

Re: [HACKERS] corrupt pages detected by enabling checksums

On Wed, 2013-05-01 at 11:33 -0400, Robert Haas wrote: > >> The only time the VM and the data page are out of sync during vacuum is > >> after a crash, right? If that's the case, I didn't think it was a big > >> deal to dirty one extra page (should be extremely rare). Am I missing > >> something? >

Re: [HACKERS] corrupt pages detected by enabling checksums

On 1 May 2013 19:16, Robert Haas wrote: > On Wed, May 1, 2013 at 1:02 PM, Simon Riggs wrote: >> I agree, but that was in the original coding wasn't it? > > I believe the problem was introduced by this commit: > > commit fdf9e21196a6f58c6021c967dc5776a16190f295 > Author: Heikki Linnakangas > Date

Re: [HACKERS] Documentation epub format

On 05/01/2013 10:52 AM, Tom Lane wrote: > "Joshua D. Drake" writes: >> Once upon a time we had multiple books as documentation, then at some >> point we merged them. It was quite a few years ago. >> I would agree at this point that we need to consider breaking them up >> again. The documentation

Re: [HACKERS] Documentation epub format

> Also the divisions between sections were totally arbitrary and unintuitive. > > I think it would make a lot more sense to modify the SGML export to > create a book per chapter. Also ... why is this discussion not on pgsql-docs, where it belongs? Crossing it over. -- Josh Berkus PostgreSQL Ex

Re: [HACKERS] corrupt pages detected by enabling checksums

On Wed, 2013-05-01 at 14:16 -0400, Robert Haas wrote: > Now that I'm looking at this, I'm a bit confused by the new logic in > visibilitymap_set(). When checksums are enabled, we set the page LSN, > which is described like this: "we need to protect the heap page from > being torn". But how does s

Re: [HACKERS] corrupt pages detected by enabling checksums

On Wed, 2013-05-01 at 20:06 +0100, Simon Riggs wrote: > >> Why aren't we writing just one WAL record for this action? ... > > > > I thought about that, too. It certainly seems like more than we want > > to try to do for 9.3 at this point. The other complication is that > > there's a lot of cond

Re: [HACKERS] Recovery target 'immediate'

On Fri, Apr 26, 2013 at 09:48:48AM -0400, Tom Lane wrote: > Magnus Hagander writes: > > That said, maybe the easier choice for a *system* (such as v-thingy) > > would be to simply to the full backup using pg_basebackup -x (or > > similar), therefor not needing the log archive at all when restoring

Re: [HACKERS] Graph datatype addition

On Wednesday, May 1, 2013, Atri Sharma wrote: > Hi all, > > Please find a probable prototype for the same: > > struct GraphNode > { > Oid NodeOid;// Oid of the row which is the node here. We will > store an identifier to it here rather than the complete row(with data) > itself. > Adjac

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

On Wed, May 1, 2013 at 12:10 PM, Simon Riggs wrote: > Heikki said "...remove pg_ctl -m fast/smart option altogether. There > is no need to expose that to users." > > So it is no longer exposed to users. If there are others that share > that opinion we may change this. /me blinks. I'm not sure we

Re: [HACKERS] high io BUT huge amount of free memory

On Wed, Apr 24, 2013 at 08:39:09AM -0500, Shaun Thomas wrote: > On 04/24/2013 08:24 AM, Robert Haas wrote: > > >Are you referring to the fact that vm.zone_reclaim_mode = 1 is an > >idiotic default? > > Well... it is. But even on systems where it's not the default or is > explicitly disabled, ther

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

On Wed, May 1, 2013 at 07:23:33PM -0400, Robert Haas wrote: > On Wed, May 1, 2013 at 12:10 PM, Simon Riggs wrote: > > Heikki said "...remove pg_ctl -m fast/smart option altogether. There > > is no need to expose that to users." > > > > So it is no longer exposed to users. If there are others that

Re: [HACKERS] Proposal to add --single-row to psql

On Mon, Apr 29, 2013 at 4:33 PM, Jim Nasby wrote: > On 4/28/13 7:50 AM, Craig Ringer wrote: >> >> I find it frustrating that I've never seen an @paraccel email address here >> and that few of the other vendors of highly customised Pg offshoots are >> contributing back. It's almost enough to make m

[HACKERS] Re: [BUGS] BUG #8128: pg_dump (>= 9.1) failed while dumping a scheme named "old" from PostgreSQL 8.4

On Wed, May 1, 2013 at 6:38 PM, Tom Lane wrote: > This is complete nonsense, because David's argument is pretty clearly not nonsense. I think they're valid well reasoned arguments. It's just that the evidence is mixed and on balance leans towards not unnecessarily reserving keywords. Fwiw we've

Re: [HACKERS] Proposal to add --single-row to psql

> I find it frustrating that I've never seen an @paraccel email address > here and that few of the other vendors of highly customised Pg offshoots > are contributing back. It's almost enough to make me like the GPL. Well, Paraccel never ended up contributing any code, but in years back (2006-2008

Re: [HACKERS] Documentation epub format

On Wed, May 1, 2013 at 5:27 PM, Fabien COELHO wrote: > I must admit that there is a bit of a disappointement as far as the user > experience is concerned: the generated file is barely usable on an iPad2 > with the default iBooks reader, which was clearly not designed for handling > a "4592" pages

Re: [HACKERS] Proposal to add --single-row to psql

On 05/02/2013 08:36 AM, Robert Haas wrote: > I've talked with a bunch of other companies over the last few years > who also chose PostgreSQL for licensing reasons. I'm pretty happy > about that. I think it's a pretty good thing too, personally ... but I do wish they'd contribute a bit more to th

Re: [HACKERS] Proposal to add --single-row to psql

Robert Haas writes: > ... There are in fact a > pretty large number of companies - EnterpriseDB, obviously, but there > are many, many others - that are choosing to build businesses around > PostgreSQL precisely because it *isn't* GPL. Personally, I think > that's a good thing for our community i

Re: [HACKERS] Graph datatype addition

Sent from my iPad On 02-May-2013, at 4:33, Misa Simic wrote: > > > On Wednesday, May 1, 2013, Atri Sharma wrote: >> Hi all, >> >> Please find a probable prototype for the same: >> >> struct GraphNode >> { >> Oid NodeOid;// Oid of the row which is the node here. We will >> store an

[HACKERS] Archive Recovery and SR promote command is failed by “contrecord is requested” in ReadRecord()

Hi, I found that archive recovery and SR promote command is failed by contrecord is requested by 0/420" in ReadRecord(). I investigate about "contrecord", it means that record crosses page boundary. I think it is not irregular page, and should be try to read next page in this case. But in a

Re: [HACKERS] Documentation epub format

On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: > I must admit that there is a bit of a disappointement as far as the > user experience is concerned: the generated file is barely usable on > an iPad2 with the default iBooks reader, which was clearly not > designed for handling a "4592" page

Re: [HACKERS] Documentation epub format

On 02/05/13 15:23, Peter Eisentraut wrote: On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with the default iBooks reader, which was clearly n

[HACKERS] \watch stuck on execution of commands returning no tuples

Hi all, When testing \watch, I noticed that process waits indefinitely when executing it with a DDL or a DML. For example: postgres=# CREATE TABLE aa (a int); postgres=# ANALYSE aa \watch 10 -- Process waiting here By referring at do_watch:command.c, the feature is made such as there is no error

Re: [HACKERS] Recovery target 'immediate'

On Thu, May 2, 2013 at 7:40 AM, Bruce Momjian wrote: > On Fri, Apr 26, 2013 at 09:48:48AM -0400, Tom Lane wrote: > > Magnus Hagander writes: > > > That said, maybe the easier choice for a *system* (such as v-thingy) > > > would be to simply to the full backup using pg_basebackup -x (or > > > sim

[HACKERS] Confusing comment in xlog.c or am I missing something?

I was just going through the xlog.c and I came across following which confused me: Given, src/include/access/xlogdefs.h #define XLogSegsPerFile (((uint32) 0x) / XLogSegSize) #define XLogFileSize(XLogSegsPerFile * XLogSegSize) Also, typedef struct XLogRecPtr { uint32

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

On 02.05.2013 08:16, Amit Langote wrote: I was just going through the xlog.c and I came across following which confused me: Given, src/include/access/xlogdefs.h #define XLogSegsPerFile (((uint32) 0x) / XLogSegSize) #define XLogFileSize(XLogSegsPerFile * XLogSegSize) Also, typedef

[HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

Hi All, While looking at the help message of pg_basebackup, I got a bit confused by it. It looks something like this: Connection options: -d, --dbname=CONNSTR connection string So the long option says "--dbname" which is not really what its expecting. The same issue also applies to pg_receiv

Re: [HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

On 02.05.2013 08:43, Pavan Deolasee wrote: Hi All, While looking at the help message of pg_basebackup, I got a bit confused by it. It looks something like this: Connection options: -d, --dbname=CONNSTR connection string So the long option says "--dbname" which is not really what its expec

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

> > /* Check for crossing of xlog segment boundary */ > > if (RecPtr->xrecoff>= XLogFileSize) > > { > > (RecPtr->xlogid)++; > > RecPtr->xrecoff = 0; > > } > > > > Is that xlog "file" boundary or am I missing something? > > The WAL space is divided into 4GB "log files", which are n

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

On 02.05.2013 09:11, Amit Langote wrote: In that case, should the comment be "/* Check for crossing of xlog file boundary */" instead of /* Check for crossing of xlog segment boundary */, since ( RecPtr->xrecoff>= XLogFileSize ) would mean crossing the xlog "file" (not segment) boundary, right?

Re: [HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

On Thu, May 2, 2013 at 11:24 AM, Heikki Linnakangas wrote: > >> >> > In other tools too, like psql, you can pass a connection string with > -d/--dbname, which is why I thought it would be best to use the same option > for passing a connection string to pg_basebackup/pg_dumpall too. > > Ah Ok. I w

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

> Yeah, that would be more correct. The phrase we seem to use elsewhere in > xlog.c is "crossing a logid boundary". Should we change it in 9.2 to clear the confusion? (Attached is a rather small patch to fix that! :) ) -- Amit Langote minor-xlog-comment.patch (914 bytes)

Re: [HACKERS] Recovery target 'immediate'

On 26 April 2013 18:13, Heikki Linnakangas wrote: > On 26.04.2013 19:50, Magnus Hagander wrote: >> >> On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggs >> wrote: >>> >>> On 26 April 2013 17:25, Heikki Linnakangas >>> wrote: Actually, from a usability point of view I think would be nice to hav

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

On Thu, May 2, 2013 at 9:25 AM, Bruce Momjian wrote: > On Wed, May 1, 2013 at 07:23:33PM -0400, Robert Haas wrote: >> On Wed, May 1, 2013 at 12:10 PM, Simon Riggs wrote: >> > Heikki said "...remove pg_ctl -m fast/smart option altogether. There >> > is no need to expose that to users." >> > >> >