Re: [HACKERS] Using ini file to setup replication
On Mon, Jul 22, 2013 at 10:59 PM, Greg Stark st...@mit.edu wrote: On Fri, Jul 19, 2013 at 2:20 PM, Samrat Revagade revagade.sam...@gmail.com wrote: for example: if i want to configure 2 servers then it will add 6 lines,for 3 -9, for 4-12 setting's for particular server will be: considering the way of setting value to conf parameters : guc . value standby_name.'AAA' synchronous_transfer. commit wal_sender_timeout.60 I have a feeling Samrat and Sawada-san have some good use cases where this extra syntax could be a big step up in expressiveness. But I'm having a hard time figuring out exactly what they have in mind. If either of you could explain in more detail how the extra syntax would apply to your use case and how it would let you do something that you can't already do it might be helpful. I'm assuming the idea is something like having a single config file which can work for the server regardless of whether it's acting as the primary or standby and then be able to switch roles by switching a single flag which would control which parts of the config file were applied. But I'm having a hard time seeing how exactly that would work. in this approach which GUC parameters is written in postgresql.conf, user would write many extra line in postgresql.conf by a standby server as Samrat suggested. It will increase size of postgresql.conf. I think it is not good that all those parameters are written in postgresql.conf. that is, I think that those parameters should be written in separately file. e.g., we can set separately any parameter using include (or include if exist) in postgresql.conf. if include file doesn't exist, we would set default value to each wal sender. that is, we give up ini file, and we provide mechanism of setting to each wal sender as option of overwrite. of course to support this approach, it needs to use the patch which Andres suggested, and server should be able to handle toke which is two or mote separated by a dot. so we would like to help this approach. Regards, Sawada Masahiko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using ini file to setup replication
On Fri, Jul 19, 2013 at 6:53 PM, Andres Freund and...@2ndquadrant.com wrote: So you can just do stuff like: server.foo.grand_unified_config = value. it looks good to me too. when server parse values which is written in postgresql.conf, server handles those parameter as item list value. after that, those parameter overwrite to corresponding values. so, we can allocate area of those values dynamically when server first parse those values. (of cause we should count number of servers) but I think that when server setting value is no set in order by server name, parsing is a little difficult. we should check whether server name has already registered at all time. for example : postgresql.conf - server.AAA.wal_sender_timeout = 60 server.BBB.synchronous_transfer = data_flush server.AAA.synchronous_transfer = all postgresql.conf - what do you think? and if the parameter(e.g., wal_sender_timeout) is not set in configure file(or postgresql,conf) like above, what value should we set value to parameter? and how we handle originally wal_sender_timeout? will it leave? or delete? Regards, --- Sawada Masahiko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using ini file to setup replication
On Fri, Jul 19, 2013 at 2:20 PM, Samrat Revagade revagade.sam...@gmail.com wrote: for example: if i want to configure 2 servers then it will add 6 lines,for 3 -9, for 4-12 setting's for particular server will be: considering the way of setting value to conf parameters : guc . value standby_name.'AAA' synchronous_transfer. commit wal_sender_timeout.60 I have a feeling Samrat and Sawada-san have some good use cases where this extra syntax could be a big step up in expressiveness. But I'm having a hard time figuring out exactly what they have in mind. If either of you could explain in more detail how the extra syntax would apply to your use case and how it would let you do something that you can't already do it might be helpful. I'm assuming the idea is something like having a single config file which can work for the server regardless of whether it's acting as the primary or standby and then be able to switch roles by switching a single flag which would control which parts of the config file were applied. But I'm having a hard time seeing how exactly that would work. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Using ini file to setup replication
Hi, I was going through the archives and there was a discussion about using ini file to setup replication.(http://www.postgresql.org/message-id/4c9876b4.9020...@enterprisedb.com). I think if we work on this proposal and separate out the replication setup from postgresql.conf file then we can provide more granularity while setting up the replication parameters. for example, we can set different values of wal_sender_timeout for each standby sever. So i think it is good idea to separate out the replication settings from postgresql.conf file and put into ini file. Once it is confirmed then we can extend the ini file to support future developments into replication. *for example: for failback safe standby.* Below I have explained how to to use ini file for failback safe stadby setup: While discussing feature of fail-back safe standby (CAF8Q-Gy7xa60HwXc0MKajjkWFEbFDWTG=ggyu1kmt+s2xcq...@mail.gmail.com) We have decided to use the *ini* file to configure the fail-back safe standby here is the link for that: cad21aocy2_bqvpzjey7s77amnccbxfj+1gphggdbulklav0...@mail.gmail.com But there is no strong positive/negative feedback for the concept of introducing the ini file.please have a look at it and give feedback. In todays scenario, In replication we only have the 2 ways of configuring the standby server 1. Asynchronous standby 2. Synchronous standby With the patch of failback safe standby we have more granularity in setting up the standby servers. 1. Default synchronous standby.(AAA) 2. Default asynchronous standby. (BBB) 3. Synchronous standby and also make same standby as a failback safe standby.(CCC) 4. Asynchronous standby and also make same standby as a failback safe standby.(DDD) In failback safe standby we are thinking to add to more granular settings of replication parameters for example 1. User can set seperate value for wal_sender_timeout for each server. 2. User can set seperate value of synchronous_transfer for each server. Consider the scenario where user want to setup the 4 standby servers as explained above so setting for them will be: - ini file - [Server] standby_name = 'AAA' synchronous_transfer = commit wal_sender_timeout = 60 [Server] standby_name = 'BBB' synchronous_transfer = none wal_sender_timeout = 40 [Server] standby_name = 'CCC' synchronous_transfer = all wal_sender_timeout = 50 [Server] standby_name = 'DDD' synchronous_transfer = data_flush wal_sender_timeout = 50 so setting up such a scenario through postgresql.conf file is impossible and if we try to do that it will add lot of complexity to the code. so use of ini file will be the very good choice in this case. Thank you , Samrat -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using ini file to setup replication
Please find updated hyperlinks: Below I have explained how to to use ini file for failback safe stadby setup: While discussing feature of fail-back safe standby (CAF8Q-Gy7xa60HwXc0MKajjkWFEbFDWTG=ggyu1kmt+s2xcq...@mail.gmail.com) http://www.postgresql.org/message-id/CAF8Q-Gy7xa60HwXc0MKajjkWFEbFDWTG=ggyu1kmt+s2xcq...@mail.gmail.com We have decided to use the *ini* file to configure the fail-back safe standby here is the link for that: cad21aocy2_bqvpzjey7s77amnccbxfj+1gphggdbulklav0...@mail.gmail.com http://www.postgresql.org/message-id/cad21aocy2_bqvpzjey7s77amnccbxfj+1gphggdbulklav0...@mail.gmail.com Regards, Samrat Revgade -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using ini file to setup replication
On 2013-07-19 14:54:16 +0530, Samrat Revagade wrote: I was going through the archives and there was a discussion about using ini file to setup replication.(http://www.postgresql.org/message-id/4c9876b4.9020...@enterprisedb.com). I think if we work on this proposal and separate out the replication setup from postgresql.conf file then we can provide more granularity while setting up the replication parameters. for example, we can set different values of wal_sender_timeout for each standby sever. So i think it is good idea to separate out the replication settings from postgresql.conf file and put into ini file. Once it is confirmed then we can extend the ini file to support future developments into replication. *for example: for failback safe standby.* I think that introducing another configuration format is a pretty bad idea. While something new might not turn out to be as bad, we've seen how annoying a separate configuration format turned out for recovery.conf. I'd much rather go ahead and remove the nesting limit of GUCs. That should give us just about all that can be achieved with an ini file with a 1 line change. Sometime we might want to extend our format to add ini like sections but I think that *definitely* should be a separate proposal. I've even proposed that in the past in 20130225211533.gd3...@awork2.anarazel.de . I plan to propose an updated version of that patch (not allowing numeric 2nd level ids) for the next CF. So you can just do stuff like: server.foo.grand_unified_config = value. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using ini file to setup replication
*for example: for failback safe standby.* I think that introducing another configuration format is a pretty bad idea. While something new might not turn out to be as bad, we've seen how annoying a separate configuration format turned out for recovery.conf. Its not totally different way of configuration. ini file will be parsed in the same way as postgresql.conf. just want to separate out the replication parameters, to make simpler configuration for future developments in the field of replication such as failback-safe standby. So you can just do stuff like: server.foo.grand_unified_config = value. But according to your approach and considering the use case of failback safe standby the parameters into the postgresql.conf will vary dynamically, and i don't think so doing this in the postgresql.conf is a good idea because it already contains whole bunch of parameters: for example: if i want to configure 2 servers then it will add 6 lines,for 3 -9, for 4-12 setting's for particular server will be: considering the way of setting value to conf parameters : guc . value standby_name.'AAA' synchronous_transfer. commit wal_sender_timeout.60 Regards, Samrat Samrat Revgade -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using ini file to setup replication
I've even proposed that in the past in 20130225211533.gd3...@awork2.anarazel.de . I plan to propose an updated version of that patch (not allowing numeric 2nd level ids) for the next CF. So you can just do stuff like: server.foo.grand_unified_config = value. Sounds good to me. I can see lots of uses for that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
