Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-13 Thread Amit Kapila
...@huawei.com To: Dan Scales sca...@vmware.com, PG Hackers pgsql-hackers@postgresql.org Sent: Tuesday, February 7, 2012 1:08:49 AM Subject: Re: [HACKERS] double writes using double-write buffer approach [WIP] I think it is a good idea, and can help double-writes perform better in the case of lots

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-08 Thread Dan Scales
Scales sca...@vmware.com, PG Hackers pgsql-hackers@postgresql.org Sent: Tuesday, February 7, 2012 1:08:49 AM Subject: Re: [HACKERS] double writes using double-write buffer approach [WIP] I think it is a good idea, and can help double-writes perform better in the case of lots of backend

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-07 Thread Amit Kapila
I think it is a good idea, and can help double-writes perform better in the case of lots of backend evictions. I don't understand this point, because from the data in your mail, it appears that when shared buffers are less means when more evictions can happen, the performance is less.

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-07 Thread Greg Smith
On 02/07/2012 12:09 AM, Dan Scales wrote: So, yes, good point -- double writes cannot replace the functionality of full_page_writes for base backup. If double writes were in use, they might be automatically switched over to full page writes for the duration of the base backup. And the

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-06 Thread Fujii Masao
On Sat, Jan 28, 2012 at 7:31 AM, Dan Scales sca...@vmware.com wrote: Let me know if you have any thoughts/comments, etc.  The patch is enclosed, and the README.doublewrites is updated a fair bit. ISTM that the double-write can prevent torn-pages in neither double-write file nor data file in

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-06 Thread Dan Scales
pgsql-hackers@postgresql.org Sent: Monday, February 6, 2012 3:08:15 AM Subject: Re: [HACKERS] double writes using double-write buffer approach [WIP] On Sat, Jan 28, 2012 at 7:31 AM, Dan Scales sca...@vmware.com wrote: Let me know if you have any thoughts/comments, etc.  The patch is enclosed

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-05 Thread Dan Scales
on. Thanks, Dan - Original Message - From: Robert Haas robertmh...@gmail.com To: Dan Scales sca...@vmware.com Cc: PG Hackers pgsql-hackers@postgresql.org Sent: Friday, February 3, 2012 1:48:54 PM Subject: Re: [HACKERS] double writes using double-write buffer approach [WIP] On Fri, Feb 3

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-05 Thread Robert Haas
On Sun, Feb 5, 2012 at 4:17 PM, Dan Scales sca...@vmware.com wrote: Thanks for the detailed followup.  I do see how Postgres is tuned for having a bunch of memory available that is not in shared_buffers, both for the OS buffer cache and other memory allocations.  However, Postgres seems to run

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-03 Thread Dan Scales
Message - From: Robert Haas robertmh...@gmail.com To: Dan Scales sca...@vmware.com Cc: PG Hackers pgsql-hackers@postgresql.org Sent: Thursday, February 2, 2012 7:19:47 AM Subject: Re: [HACKERS] double writes using double-write buffer approach [WIP] On Fri, Jan 27, 2012 at 5:31 PM, Dan Scales

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-03 Thread Robert Haas
On Fri, Feb 3, 2012 at 3:14 PM, Dan Scales sca...@vmware.com wrote: Thanks for the feedback!  I think you make a good point about the small size of dirty data in the OS cache.  I think what you can say about this double-write patch is that it will work not work well for configurations that

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-02 Thread Robert Haas
On Fri, Jan 27, 2012 at 5:31 PM, Dan Scales sca...@vmware.com wrote: I've been prototyping the double-write buffer idea that Heikki and Simon had proposed (as an alternative to a previous patch that only batched up writes by the checkpointer).  I think it is a good idea, and can help