Re: [HACKERS] Logging WAL when updating hintbit

2014-01-01 Thread Robert Haas
On Tue, Dec 24, 2013 at 7:31 PM, Michael Paquier michael.paqu...@gmail.com wrote: Yep, finding all the code paths with a single keyword is usually a good thing. Attached is a purely-aesthetical patch that updates the internal variable name to wal_log_hints. There are many GUC parameters other

Re: [HACKERS] Logging WAL when updating hintbit

2014-01-01 Thread Michael Paquier
On Thu, Jan 2, 2014 at 10:21 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 24, 2013 at 7:31 PM, Michael Paquier michael.paqu...@gmail.com wrote: Yep, finding all the code paths with a single keyword is usually a good thing. Attached is a purely-aesthetical patch that updates the

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-24 Thread Fujii Masao
On Tue, Dec 24, 2013 at 1:58 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Dec 21, 2013 at 4:13 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 20, 2013 at 9:06 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Michael Paquier escribió: On Fri, Dec 20, 2013 at 1:05

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-24 Thread Michael Paquier
On Wed, Dec 25, 2013 at 2:36 AM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Dec 24, 2013 at 1:58 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Dec 21, 2013 at 4:13 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 20, 2013 at 9:06 AM, Alvaro Herrera

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-23 Thread Michael Paquier
On Sat, Dec 21, 2013 at 4:13 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 20, 2013 at 9:06 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Michael Paquier escribió: On Fri, Dec 20, 2013 at 1:05 PM, Sawada Masahiko sawada.m...@gmail.com wrote: Sorry the patch which I

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-20 Thread Alvaro Herrera
Michael Paquier escribió: On Fri, Dec 20, 2013 at 1:05 PM, Sawada Masahiko sawada.m...@gmail.com wrote: Sorry the patch which I attached has wrong indent on pg_controldata. I have modified it and attached the new version patch. Now that you send this patch, I am just recalling some recent

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-20 Thread Fujii Masao
On Fri, Dec 20, 2013 at 2:05 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, Dec 20, 2013 at 3:38 AM, Sawada Masahiko sawada.m...@gmail.com wrote: On Thu, Dec 19, 2013 at 12:37 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:30 AM, Michael Paquier

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-20 Thread Robert Haas
On Fri, Dec 20, 2013 at 9:06 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Michael Paquier escribió: On Fri, Dec 20, 2013 at 1:05 PM, Sawada Masahiko sawada.m...@gmail.com wrote: Sorry the patch which I attached has wrong indent on pg_controldata. I have modified it and attached the

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-20 Thread Heikki Linnakangas
On 12/20/2013 08:34 PM, Fujii Masao wrote: On Fri, Dec 20, 2013 at 2:05 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, Dec 20, 2013 at 3:38 AM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached the patch which changes name from 'wal_log_hintbits' to 'wal_log_hints'. It gained

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-19 Thread Sawada Masahiko
On Thu, Dec 19, 2013 at 12:37 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:30 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:22 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Dec 13, 2013 at 7:57 PM, Heikki Linnakangas

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-19 Thread Sawada Masahiko
On Fri, Dec 20, 2013 at 3:38 AM, Sawada Masahiko sawada.m...@gmail.com wrote: On Thu, Dec 19, 2013 at 12:37 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:30 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:22 AM, Amit Kapila

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-19 Thread Michael Paquier
On Fri, Dec 20, 2013 at 1:05 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, Dec 20, 2013 at 3:38 AM, Sawada Masahiko sawada.m...@gmail.com wrote: On Thu, Dec 19, 2013 at 12:37 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:30 AM, Michael Paquier

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 10:22 PM, Amit Kapila amit.kapil...@gmail.com wrote: Me either; at the very least, it's short an underscore: wal_log_hint_bits would be more readable. But how about just wal_log_hints? +1 for wal_log_hints, it sounds better. +1. -- Robert Haas EnterpriseDB:

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Sawada Masahiko
2013/12/14 0:14 Tom Lane t...@sss.pgh.pa.us: Heikki Linnakangas hlinnakan...@vmware.com writes: I'm not totally satisfied with the name of the GUC, wal_log_hintbits. Me either; at the very least, it's short an underscore: wal_log_hint_bits would be more readable. But how about just

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Amit Kapila
On Wed, Dec 18, 2013 at 11:30 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 18, 2013 at 11:22 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Dec 13, 2013 at 7:57 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Thanks, committed with some minor changes: Should

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-17 Thread Amit Kapila
On Fri, Dec 13, 2013 at 7:57 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/13/2013 07:55 AM, Sawada Masahiko wrote: On Fri, Dec 13, 2013 at 1:51 PM, Dilip kumar dilip.ku...@huawei.com wrote: On 04 December 2013, Sawada Masahiko Wrote I have modified the patch base on your

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-17 Thread Michael Paquier
On Wed, Dec 18, 2013 at 11:22 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Dec 13, 2013 at 7:57 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Thanks, committed with some minor changes: Should this patch in CF app be moved to Committed Patches or is there something left for

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-13 Thread Dilip kumar
On Fri, Dec 13, 2013 at 11:25, Sawada Masahiko Wrote I attached the patch which have modified based on Robert suggestion, and fixed typo. I have reviewed the modified patch and I have some comments.. 1. Patch need to be rebased (failed applying on head) 2. crc field should be at

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-13 Thread Sawada Masahiko
On Fri, Dec 13, 2013 at 5:49 PM, Dilip kumar dilip.ku...@huawei.com wrote: On Fri, Dec 13, 2013 at 11:25, Sawada Masahiko Wrote I attached the patch which have modified based on Robert suggestion, and fixed typo. I have reviewed the modified patch and I have some comments.. 1. Patch

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-13 Thread Heikki Linnakangas
On 12/13/2013 07:55 AM, Sawada Masahiko wrote: On Fri, Dec 13, 2013 at 1:51 PM, Dilip kumar dilip.ku...@huawei.com wrote: On 04 December 2013, Sawada Masahiko Wrote I attached the patch which have modified based on Robert suggestion, and fixed typo. I have reviewed the modified patch and I

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-13 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: I'm not totally satisfied with the name of the GUC, wal_log_hintbits. Me either; at the very least, it's short an underscore: wal_log_hint_bits would be more readable. But how about just wal_log_hints? regards, tom

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-13 Thread Bruce Momjian
On Fri, Dec 13, 2013 at 10:14:06AM -0500, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: I'm not totally satisfied with the name of the GUC, wal_log_hintbits. Me either; at the very least, it's short an underscore: wal_log_hint_bits would be more readable. But how

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-12 Thread Dilip kumar
On 04 December 2013, Sawada Masahiko Wrote I attached the patch which have modified based on Robert suggestion, and fixed typo. I have reviewed the modified patch and I have some comments.. 1. Patch need to be rebased (failed applying on head) 2. crc field should be at end in

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-12 Thread Sawada Masahiko
On Fri, Dec 13, 2013 at 1:51 PM, Dilip kumar dilip.ku...@huawei.com wrote: On 04 December 2013, Sawada Masahiko Wrote I attached the patch which have modified based on Robert suggestion, and fixed typo. I have reviewed the modified patch and I have some comments.. 1. Patch need to be

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-03 Thread Sawada Masahiko
On Tue, Dec 3, 2013 at 4:28 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 3, 2013 at 3:30 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Tue, Dec 3, 2013 at 12:02 PM, Michael Paquier michael.paqu...@gmail.com wrote: Indeed, I forgot this code path. Completing for

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-03 Thread Sawada Masahiko
On Tue, Dec 3, 2013 at 5:34 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Tue, Dec 3, 2013 at 4:28 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 3, 2013 at 3:30 PM, Sawada Masahiko sawada.m...@gmail.com wrote: After more thinking... Before performing a rewind on a

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-02 Thread Robert Haas
On Mon, Dec 2, 2013 at 12:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Nov 28, 2013 at 9:42 PM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached new version patch which have modify typos and added documentation patch. The documentation part of patch is implemented

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-02 Thread Michael Paquier
On Tue, Dec 3, 2013 at 1:08 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Dec 2, 2013 at 12:54 AM, Michael Paquier Considering that, it would make more sense to have this option settable with initdb only and not changeable after initialization, in the same fashion as checksums. Having a

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-02 Thread Michael Paquier
On Tue, Dec 3, 2013 at 11:57 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 3, 2013 at 1:08 AM, Robert Haas robertmh...@gmail.com wrote: As long as it was set at the time the master last entered read-write mode (which is what the XLOG_PARAMETER_CHANGE stuff does) you should

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-02 Thread Sawada Masahiko
On Tue, Dec 3, 2013 at 12:02 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 3, 2013 at 11:57 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 3, 2013 at 1:08 AM, Robert Haas robertmh...@gmail.com wrote: Forcing it to be done only an initdb-time is excessive.

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-02 Thread Michael Paquier
On Tue, Dec 3, 2013 at 3:30 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Tue, Dec 3, 2013 at 12:02 PM, Michael Paquier michael.paqu...@gmail.com wrote: Indeed, I forgot this code path. Completing for saving the state and xlog_redo for replay would be enough. Wait a minute, I retract

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-01 Thread Michael Paquier
On Thu, Nov 28, 2013 at 9:42 PM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached new version patch which have modify typos and added documentation patch. The documentation part of patch is implemented by Samrat Revagade. Thanks for the new version. The documentation still has some typo:

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-28 Thread Sawada Masahiko
On Wed, Nov 27, 2013 at 11:22 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Nov 27, 2013 at 11:04 AM, Sawada Masahiko sawada.m...@gmail.com wrote: Thank you for feedback. I attached the v4 patch which have modified. Just name changed to 'wal_log_hintbtis'. Few typos in this

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-26 Thread Sawada Masahiko
On Mon, Nov 25, 2013 at 9:02 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sun, Nov 24, 2013 at 6:02 AM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2013-11-19 at 11:42 -0500, Robert Haas wrote: My take is that configuration options should be used to serve different use cases. One thing

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-26 Thread Peter Eisentraut
On 11/25/13, 7:02 AM, Sawada Masahiko wrote: I attached the new version patch which adds separated parameter 'enable_logging_hintbit'. Let's not add more parameters named enable_*. Every parameter enables something. Also, I'd be worried about confusion with other log_* and logging_*

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-26 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 11/25/13, 7:02 AM, Sawada Masahiko wrote: I attached the new version patch which adds separated parameter 'enable_logging_hintbit'. Let's not add more parameters named enable_*. Every parameter enables something. More to the point: there's a

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-26 Thread Sawada Masahiko
On Wed, Nov 27, 2013 at 2:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On 11/25/13, 7:02 AM, Sawada Masahiko wrote: I attached the new version patch which adds separated parameter 'enable_logging_hintbit'. Let's not add more parameters named enable_*.

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-26 Thread Michael Paquier
On Wed, Nov 27, 2013 at 11:04 AM, Sawada Masahiko sawada.m...@gmail.com wrote: Thank you for feedback. I attached the v4 patch which have modified. Just name changed to 'wal_log_hintbtis'. Few typos in this patch found while I quickly went through: 1) s/logging hintbit/logging of hint bits/ 2)

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-25 Thread Sawada Masahiko
On Sun, Nov 24, 2013 at 6:02 AM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2013-11-19 at 11:42 -0500, Robert Haas wrote: My take is that configuration options should be used to serve different use cases. One thing I like about postgres is that it doesn't have options for the sake of options.

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-23 Thread Jeff Davis
On Tue, 2013-11-19 at 11:42 -0500, Robert Haas wrote: On Thu, Nov 14, 2013 at 1:02 AM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached patch adds new wal_level 'all'. If wal_level is set 'all', the server logs WAL not only when wal_level is set 'hot_standby' ,but also when

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-22 Thread Dilip kumar
On 19 November 2013 22:19, Sawada Masahiko Wrote Thank you for comment. Actually, I had thought to add separate parameter. I think that he said that if you can proof that amount of WAL is almost same and without less performance same as before, you might not need to separate

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-22 Thread Dilip kumar
On 20 November 2013 22:12, Sawada Masahiko Wrote 1. Patch applies cleanly to master HEAD. 2. No Compilation Warning. 3. It works as per the patch expectation. Some Suggestion: 1. Add new WAL level (all) in comment in postgresql.conf wal_level = hot_standby

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-21 Thread Sawada Masahiko
On Thu, Nov 21, 2013 at 8:59 PM, Dilip kumar dilip.ku...@huawei.com wrote: On 20 November 2013 22:12, Sawada Masahiko Wrote 1. Patch applies cleanly to master HEAD. 2. No Compilation Warning. 3. It works as per the patch expectation. Some Suggestion: 1. Add new WAL level (all) in

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-20 Thread Sawada Masahiko
On Wed, Nov 20, 2013 at 9:19 PM, Dilip kumar dilip.ku...@huawei.com wrote: On 19 November 2013 22:19, Sawada Masahiko Wrote Thanks! I took it wrong. I think that there are quite a few difference amount of WAL. Did you test about amount of WAL size in your patch? Not yet. I will do

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-19 Thread Robert Haas
On Thu, Nov 14, 2013 at 1:02 AM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached patch adds new wal_level 'all'. If wal_level is set 'all', the server logs WAL not only when wal_level is set 'hot_standby' ,but also when updating hint bit. That is, we will be able to know all of the

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-19 Thread Sawada Masahiko
On Tue, Nov 19, 2013 at 3:54 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2013/11/15 19:27), Sawada Masahiko wrote: On Thu, Nov 14, 2013 at 7:51 PM, Florian Weimer fwei...@redhat.com wrote: On 11/14/2013 07:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-18 Thread KONDO Mitsumasa
(2013/11/15 19:27), Sawada Masahiko wrote: On Thu, Nov 14, 2013 at 7:51 PM, Florian Weimer fwei...@redhat.com wrote: On 11/14/2013 07:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Shouldn't this be a separate setting? It's useful for storage which requires

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-17 Thread Sawada Masahiko
On Fri, Nov 15, 2013 at 11:33 PM, Peter Eisentraut pete...@gmx.net wrote: On 11/14/13, 1:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Compiler warning: pg_controldata.c: In function ‘wal_level_str’: pg_controldata.c:72:2: warning: enumeration value

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-15 Thread Sawada Masahiko
On Thu, Nov 14, 2013 at 7:51 PM, Florian Weimer fwei...@redhat.com wrote: On 11/14/2013 07:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Shouldn't this be a separate setting? It's useful for storage which requires rewriting a partially written sector before it

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-15 Thread Peter Eisentraut
On 11/14/13, 1:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Compiler warning: pg_controldata.c: In function ‘wal_level_str’: pg_controldata.c:72:2: warning: enumeration value ‘WAL_LEVEL_ALL’ not handled in switch [-Wswitch] -- Sent via pgsql-hackers mailing

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-14 Thread Florian Weimer
On 11/14/2013 07:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Shouldn't this be a separate setting? It's useful for storage which requires rewriting a partially written sector before it can be read again. -- Florian Weimer / Red Hat Product Security Team --

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-14 Thread Sawada Masahiko
On Thu, Nov 14, 2013 at 3:53 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Nov 14, 2013 at 3:02 PM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached patch adds new wal_level 'all'. If wal_level is set 'all', the server logs WAL not only when wal_level is set

[HACKERS] Logging WAL when updating hintbit

2013-11-13 Thread Sawada Masahiko
Hi all, I attached patch adds new wal_level 'all'. If wal_level is set 'all', the server logs WAL not only when wal_level is set 'hot_standby' ,but also when updating hint bit. That is, we will be able to know all of the changed block number by reading the WALs. This wal_level is infrastructure

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-13 Thread Michael Paquier
On Thu, Nov 14, 2013 at 3:02 PM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached patch adds new wal_level 'all'. If wal_level is set 'all', the server logs WAL not only when wal_level is set 'hot_standby' ,but also when updating hint bit. That is, we will be able to know all of the