Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-24 Thread karavelov
- Цитат от David Fetter (da...@fetter.org), на 23.07.2012 в 15:41 - I'm not sure how you automate testing a pull-the-plug scenario. I have a dim memory of how the FreeBSD project was alleged to have done it, namely by rigging a serial port (yes, it was that long ago) to the pow

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Jeff Janes
On Mon, Jul 23, 2012 at 5:41 AM, David Fetter wrote: > On Mon, Jul 23, 2012 at 08:29:16AM -0400, Andrew Dunstan wrote: >> >> >> I'm not sure how you automate testing a pull-the-plug scenario. > > I have a dim memory of how the FreeBSD project was alleged to have > done it, namely by rigging a seri

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Craig Ringer
On 07/23/2012 09:47 PM, Andrew Dunstan wrote: On 07/23/2012 09:04 AM, Craig Ringer wrote: On 07/23/2012 08:29 PM, Andrew Dunstan wrote: I'm not sure how you automate testing a pull-the-plug scenario. fire up kvm or qemu instances, then kill 'em. Yeah, maybe. Knowing just when to kill th

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 09:04 AM, Craig Ringer wrote: On 07/23/2012 08:29 PM, Andrew Dunstan wrote: I'm not sure how you automate testing a pull-the-plug scenario. fire up kvm or qemu instances, then kill 'em. Yeah, maybe. Knowing just when to kill them might be an interesting question. I'm als

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Craig Ringer
On 07/23/2012 08:29 PM, Andrew Dunstan wrote: I'm not sure how you automate testing a pull-the-plug scenario. fire up kvm or qemu instances, then kill 'em. -- Craig Ringer -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.p

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 08:41 AM, David Fetter wrote: The buildfarm is not at all designed to test performance. That's why we want a performance farm. Right. Apart from hardware, what are we stalled on? Software :-) I am trying to find some cycles to get something going. cheers andrew -- Sent v

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread David Fetter
On Mon, Jul 23, 2012 at 08:29:16AM -0400, Andrew Dunstan wrote: > > On 07/23/2012 12:37 AM, David Fetter wrote: > >On Tue, Jul 17, 2012 at 06:56:50PM -0400, Tom Lane wrote: > >>Robert Haas writes: > >>>On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane wrote: > BTW, while we are on the subject: hasn'

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 12:37 AM, David Fetter wrote: On Tue, Jul 17, 2012 at 06:56:50PM -0400, Tom Lane wrote: Robert Haas writes: On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane wrote: BTW, while we are on the subject: hasn't this split completely broken the statistics about backend-initiated writes? Ye

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-22 Thread David Fetter
On Tue, Jul 17, 2012 at 06:56:50PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane wrote: > >> BTW, while we are on the subject: hasn't this split completely > >> broken the statistics about backend-initiated writes? > > > Yes, it seems to have done jus

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-18 Thread Simon Riggs
On 17 July 2012 23:56, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane wrote: >>> BTW, while we are on the subject: hasn't this split completely broken >>> the statistics about backend-initiated writes? > >> Yes, it seems to have done just that. > > So I went t

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-18 Thread Heikki Linnakangas
On 18.07.2012 02:48, Peter Geoghegan wrote: On 17 July 2012 23:56, Tom Lane wrote: This implies that nobody has done pull-the-plug testing on either HEAD or 9.2 since the checkpointer split went in (2011-11-01), because even a modicum of such testing would surely have shown that we're failing t

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Tom Lane
Greg Smith writes: > On 07/17/2012 06:56 PM, Tom Lane wrote: >> Furthermore, I would say that any performance testing done since then, >> if it wasn't looking at purely read-only scenarios, isn't worth the >> electrons it's written on. In particular, any performance gain that >> anybody might hav

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Craig Ringer
On 07/18/2012 12:00 PM, Greg Smith wrote: The second justification for the split was that it seems easier to get a low power result from, which I believe was the angle Peter Geoghegan was working when this popped up originally. The checkpointer has to run sometimes, but only at a 50% duty cyc

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Greg Smith
On 07/17/2012 06:56 PM, Tom Lane wrote: So I went to fix this in the obvious way (attached), but while testing it I found that the number of buffers_backend events reported during a regression test run barely changed; which surprised the heck out of me, so I dug deeper. The cause turns out to be

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Peter Geoghegan
On 17 July 2012 23:56, Tom Lane wrote: > This implies that nobody has done pull-the-plug testing on either HEAD > or 9.2 since the checkpointer split went in (2011-11-01), because even > a modicum of such testing would surely have shown that we're failing to > fsync a significant fraction of our w

[HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Tom Lane
Robert Haas writes: > On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane wrote: >> BTW, while we are on the subject: hasn't this split completely broken >> the statistics about backend-initiated writes? > Yes, it seems to have done just that. So I went to fix this in the obvious way (attached), but whil