Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-24 Thread karavelov
- Цитат от David Fetter (da...@fetter.org), на 23.07.2012 в 15:41 - I'm not sure how you automate testing a pull-the-plug scenario. I have a dim memory of how the FreeBSD project was alleged to have done it, namely by rigging a serial port (yes, it was that long ago) to the

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 12:37 AM, David Fetter wrote: On Tue, Jul 17, 2012 at 06:56:50PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, while we are on the subject: hasn't this split completely broken the

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread David Fetter
On Mon, Jul 23, 2012 at 08:29:16AM -0400, Andrew Dunstan wrote: On 07/23/2012 12:37 AM, David Fetter wrote: On Tue, Jul 17, 2012 at 06:56:50PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, while we

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 08:41 AM, David Fetter wrote: The buildfarm is not at all designed to test performance. That's why we want a performance farm. Right. Apart from hardware, what are we stalled on? Software :-) I am trying to find some cycles to get something going. cheers andrew -- Sent

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Craig Ringer
On 07/23/2012 08:29 PM, Andrew Dunstan wrote: I'm not sure how you automate testing a pull-the-plug scenario. fire up kvm or qemu instances, then kill 'em. -- Craig Ringer -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 09:04 AM, Craig Ringer wrote: On 07/23/2012 08:29 PM, Andrew Dunstan wrote: I'm not sure how you automate testing a pull-the-plug scenario. fire up kvm or qemu instances, then kill 'em. Yeah, maybe. Knowing just when to kill them might be an interesting question. I'm

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Craig Ringer
On 07/23/2012 09:47 PM, Andrew Dunstan wrote: On 07/23/2012 09:04 AM, Craig Ringer wrote: On 07/23/2012 08:29 PM, Andrew Dunstan wrote: I'm not sure how you automate testing a pull-the-plug scenario. fire up kvm or qemu instances, then kill 'em. Yeah, maybe. Knowing just when to kill

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-23 Thread Jeff Janes
On Mon, Jul 23, 2012 at 5:41 AM, David Fetter da...@fetter.org wrote: On Mon, Jul 23, 2012 at 08:29:16AM -0400, Andrew Dunstan wrote: I'm not sure how you automate testing a pull-the-plug scenario. I have a dim memory of how the FreeBSD project was alleged to have done it, namely by rigging

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-22 Thread David Fetter
On Tue, Jul 17, 2012 at 06:56:50PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, while we are on the subject: hasn't this split completely broken the statistics about backend-initiated writes?

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-18 Thread Heikki Linnakangas
On 18.07.2012 02:48, Peter Geoghegan wrote: On 17 July 2012 23:56, Tom Lanet...@sss.pgh.pa.us wrote: This implies that nobody has done pull-the-plug testing on either HEAD or 9.2 since the checkpointer split went in (2011-11-01), because even a modicum of such testing would surely have shown

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-18 Thread Simon Riggs
On 17 July 2012 23:56, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, while we are on the subject: hasn't this split completely broken the statistics about backend-initiated writes? Yes, it

[HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, while we are on the subject: hasn't this split completely broken the statistics about backend-initiated writes? Yes, it seems to have done just that. So I went to fix this in

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Peter Geoghegan
On 17 July 2012 23:56, Tom Lane t...@sss.pgh.pa.us wrote: This implies that nobody has done pull-the-plug testing on either HEAD or 9.2 since the checkpointer split went in (2011-11-01), because even a modicum of such testing would surely have shown that we're failing to fsync a significant

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Greg Smith
On 07/17/2012 06:56 PM, Tom Lane wrote: So I went to fix this in the obvious way (attached), but while testing it I found that the number of buffers_backend events reported during a regression test run barely changed; which surprised the heck out of me, so I dug deeper. The cause turns out to

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Craig Ringer
On 07/18/2012 12:00 PM, Greg Smith wrote: The second justification for the split was that it seems easier to get a low power result from, which I believe was the angle Peter Geoghegan was working when this popped up originally. The checkpointer has to run sometimes, but only at a 50% duty

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: On 07/17/2012 06:56 PM, Tom Lane wrote: Furthermore, I would say that any performance testing done since then, if it wasn't looking at purely read-only scenarios, isn't worth the electrons it's written on. In particular, any performance gain that