Re: [HACKERS] Behaviour of take over the synchronous replication
On Wed, Aug 28, 2013 at 10:59 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Aug 27, 2013 at 4:51 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sun, Aug 25, 2013 at 3:21 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Sat, Aug 24, 2013 at 2:46 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sat, Aug 24, 2013 at 3:14 AM, Josh Berkus j...@agliodbs.com wrote: On 08/23/2013 12:42 AM, Sawada Masahiko wrote: in case (a), those priority is clear. So I think that re-taking over is correct behaviour. OHOT, in case (b), even if AAA and BBB are set same priority, AAA server steals SYNC replication. I think it is better that BBB server continue behaviour SYNC standby, and AAA should become potential server. So, you're saying that: 1) synchronous_standby_names = '*' 2) replica 'BBB' is the current sync standby 3) replica 'AAA' comes online 4) replica 'AAA' grabs sync status ? I'm sorry that you are confuse. It means that 1) synchronous_standby_names = '*' 2) replica 'AAA' is the current sync standby 3) replica 'BBB' is the current async standby (potential sync standby) 4) replica 'AAA' fail. after that, replica 'BBB' is current sync standby. 5) replica 'AAA' comes online 6) replica 'AAA' grabs sync status If that's the case, I'm not really sure that's undesirable behavior. One could argue fairly persuasively that if you care about the precendence order of sync replicas, you shouldn't use '*'. And the rule of if using *, the lowest-sorted replica name has sync is actually a predictable, easy-to-understand rule. So if you want to make this a feature request, you'll need to come up with an argument as to why the current behavior is bad. Otherwise, you're just asking us to document it better (which is a good idea). It is not depend on name of standby server. That is, The standby server, which was connected to the master server during initial configration replication, is top priority even if priority of two server are same. What is happening here is that incase of '*' as priority of both are same, system will choose whichever comes in list of registered standby's first (list is maintained in structure WalSndCtl). Each standby is registered with WalSndCtl when a new WALSender is started in function InitWalSenderSlot(). As 'AAA' has been registered first it becomes preferred sync standby even if priorities of both are same. When 'AAA' goes down, it marks that Slot entry as free (by setting pid=0 in function WalSndKill), now when 'AAA' comes back again, it gets that free Slot entry and again becomes preferred sync standby. Now if we want to fix as you are suggesting which I don't think is necessary, we might need to change WalSndKill and some other place so that whenever any standby goes down, it changes slots for already registered standby's. User must remember that which standby server connected to master server at first. I think that this behavior confuse user. so I think that we need to modify this behaviour or if '*' is used, priority of server is not same (modifying manual is also good). Here user has done the settings (setting synchronous_standby_names = '*'), after which he will not have any control which standby will become sync standby, so ideally he should not complain. It might be case that for some users current behavior is good enough which means that with '*' whichever standby has become sync standby first, it will be the sync standby always if alive. I'm thinking that it is not necessary to change WalSndKill. For example, we add the value (e.g., sync_standby) which have that which wal sender is active SYNC rep. And if sync_standby is already set and it is active, server doesn't looking for active standby. Only if sync_standby is not set and it is inactive, server looking for that which server is active SYNC rep. If so, we also prevent to find active SYNC rep whenever SyncRepReleaseWaiters() is called. For '*' case, it will be okay, but when the user has given proper names, in that case even if there is any active Sync Rep, it has to be changed based on priority. I think here how to provide a fix, so that behavior gets changed to what you describe is a second priority work, first is to show the value of use-case. Do you really know where people actually setup using '*' as configuration and if yes, are they annoyed with current behavior? I have thought about it, but could imagine a scenario where people will be using '*' in their production configurations, may be it will be useful in test labs. Thank you for your feedback. I have implemented the patch which change how to put priority on each walsender, based on I suggested. I added sync_standby value into WalCtl value. This value has that which walsender is active sync rep. This patch handle also for case that user has given proper names. Regards, --- Sawada Masahiko Sync_standby_priority_v1.patch
Re: [HACKERS] Behaviour of take over the synchronous replication
On Tue, Aug 27, 2013 at 4:51 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sun, Aug 25, 2013 at 3:21 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Sat, Aug 24, 2013 at 2:46 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sat, Aug 24, 2013 at 3:14 AM, Josh Berkus j...@agliodbs.com wrote: On 08/23/2013 12:42 AM, Sawada Masahiko wrote: in case (a), those priority is clear. So I think that re-taking over is correct behaviour. OHOT, in case (b), even if AAA and BBB are set same priority, AAA server steals SYNC replication. I think it is better that BBB server continue behaviour SYNC standby, and AAA should become potential server. So, you're saying that: 1) synchronous_standby_names = '*' 2) replica 'BBB' is the current sync standby 3) replica 'AAA' comes online 4) replica 'AAA' grabs sync status ? I'm sorry that you are confuse. It means that 1) synchronous_standby_names = '*' 2) replica 'AAA' is the current sync standby 3) replica 'BBB' is the current async standby (potential sync standby) 4) replica 'AAA' fail. after that, replica 'BBB' is current sync standby. 5) replica 'AAA' comes online 6) replica 'AAA' grabs sync status If that's the case, I'm not really sure that's undesirable behavior. One could argue fairly persuasively that if you care about the precendence order of sync replicas, you shouldn't use '*'. And the rule of if using *, the lowest-sorted replica name has sync is actually a predictable, easy-to-understand rule. So if you want to make this a feature request, you'll need to come up with an argument as to why the current behavior is bad. Otherwise, you're just asking us to document it better (which is a good idea). It is not depend on name of standby server. That is, The standby server, which was connected to the master server during initial configration replication, is top priority even if priority of two server are same. What is happening here is that incase of '*' as priority of both are same, system will choose whichever comes in list of registered standby's first (list is maintained in structure WalSndCtl). Each standby is registered with WalSndCtl when a new WALSender is started in function InitWalSenderSlot(). As 'AAA' has been registered first it becomes preferred sync standby even if priorities of both are same. When 'AAA' goes down, it marks that Slot entry as free (by setting pid=0 in function WalSndKill), now when 'AAA' comes back again, it gets that free Slot entry and again becomes preferred sync standby. Now if we want to fix as you are suggesting which I don't think is necessary, we might need to change WalSndKill and some other place so that whenever any standby goes down, it changes slots for already registered standby's. User must remember that which standby server connected to master server at first. I think that this behavior confuse user. so I think that we need to modify this behaviour or if '*' is used, priority of server is not same (modifying manual is also good). Here user has done the settings (setting synchronous_standby_names = '*'), after which he will not have any control which standby will become sync standby, so ideally he should not complain. It might be case that for some users current behavior is good enough which means that with '*' whichever standby has become sync standby first, it will be the sync standby always if alive. I'm thinking that it is not necessary to change WalSndKill. For example, we add the value (e.g., sync_standby) which have that which wal sender is active SYNC rep. And if sync_standby is already set and it is active, server doesn't looking for active standby. Only if sync_standby is not set and it is inactive, server looking for that which server is active SYNC rep. If so, we also prevent to find active SYNC rep whenever SyncRepReleaseWaiters() is called. For '*' case, it will be okay, but when the user has given proper names, in that case even if there is any active Sync Rep, it has to be changed based on priority. I think here how to provide a fix, so that behavior gets changed to what you describe is a second priority work, first is to show the value of use-case. Do you really know where people actually setup using '*' as configuration and if yes, are they annoyed with current behavior? I have thought about it, but could imagine a scenario where people will be using '*' in their production configurations, may be it will be useful in test labs. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Behaviour of take over the synchronous replication
On Sun, Aug 25, 2013 at 3:21 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Sat, Aug 24, 2013 at 2:46 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sat, Aug 24, 2013 at 3:14 AM, Josh Berkus j...@agliodbs.com wrote: On 08/23/2013 12:42 AM, Sawada Masahiko wrote: in case (a), those priority is clear. So I think that re-taking over is correct behaviour. OHOT, in case (b), even if AAA and BBB are set same priority, AAA server steals SYNC replication. I think it is better that BBB server continue behaviour SYNC standby, and AAA should become potential server. So, you're saying that: 1) synchronous_standby_names = '*' 2) replica 'BBB' is the current sync standby 3) replica 'AAA' comes online 4) replica 'AAA' grabs sync status ? I'm sorry that you are confuse. It means that 1) synchronous_standby_names = '*' 2) replica 'AAA' is the current sync standby 3) replica 'BBB' is the current async standby (potential sync standby) 4) replica 'AAA' fail. after that, replica 'BBB' is current sync standby. 5) replica 'AAA' comes online 6) replica 'AAA' grabs sync status If that's the case, I'm not really sure that's undesirable behavior. One could argue fairly persuasively that if you care about the precendence order of sync replicas, you shouldn't use '*'. And the rule of if using *, the lowest-sorted replica name has sync is actually a predictable, easy-to-understand rule. So if you want to make this a feature request, you'll need to come up with an argument as to why the current behavior is bad. Otherwise, you're just asking us to document it better (which is a good idea). It is not depend on name of standby server. That is, The standby server, which was connected to the master server during initial configration replication, is top priority even if priority of two server are same. What is happening here is that incase of '*' as priority of both are same, system will choose whichever comes in list of registered standby's first (list is maintained in structure WalSndCtl). Each standby is registered with WalSndCtl when a new WALSender is started in function InitWalSenderSlot(). As 'AAA' has been registered first it becomes preferred sync standby even if priorities of both are same. When 'AAA' goes down, it marks that Slot entry as free (by setting pid=0 in function WalSndKill), now when 'AAA' comes back again, it gets that free Slot entry and again becomes preferred sync standby. Now if we want to fix as you are suggesting which I don't think is necessary, we might need to change WalSndKill and some other place so that whenever any standby goes down, it changes slots for already registered standby's. User must remember that which standby server connected to master server at first. I think that this behavior confuse user. so I think that we need to modify this behaviour or if '*' is used, priority of server is not same (modifying manual is also good). Here user has done the settings (setting synchronous_standby_names = '*'), after which he will not have any control which standby will become sync standby, so ideally he should not complain. It might be case that for some users current behavior is good enough which means that with '*' whichever standby has become sync standby first, it will be the sync standby always if alive. I'm thinking that it is not necessary to change WalSndKill. For example, we add the value (e.g., sync_standby) which have that which wal sender is active SYNC rep. And if sync_standby is already set and it is active, server doesn't looking for active standby. Only if sync_standby is not set and it is inactive, server looking for that which server is active SYNC rep. If so, we also prevent to find active SYNC rep whenever SyncRepReleaseWaiters() is called. In pg_stat_get_walsenders(), looking for active SYNC rep is already used. I think that it is also able to here. Thought? -- Regards, --- Sawada Masahiko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Behaviour of take over the synchronous replication
Amit Kapila amit.kapil...@gmail.com wrote: What is happening here is that incase of '*' as priority of both are same, system will choose whichever comes in list of registered standby's first (list is maintained in structure WalSndCtl). Each standby is registered with WalSndCtl when a new WALSender is started in function InitWalSenderSlot(). As 'AAA' has been registered first it becomes preferred sync standby even if priorities of both are same. When 'AAA' goes down, it marks that Slot entry as free (by setting pid=0 in function WalSndKill), now when 'AAA' comes back again, it gets that free Slot entry and again becomes preferred sync standby. So, when a user says they don't care about which standby is in sync mode, the system can decide (based on internal implementation details not visible to the user) that any node which comes online could now become the sync node, and freeze commits on the source database until that newly added node catches up? Or does the existing sync node continue in that role until the new preferred node is caught up? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Behaviour of take over the synchronous replication
On Wed, Aug 28, 2013 at 5:51 AM, Kevin Grittner kgri...@ymail.com wrote: Amit Kapila amit.kapil...@gmail.com wrote: What is happening here is that incase of '*' as priority of both are same, system will choose whichever comes in list of registered standby's first (list is maintained in structure WalSndCtl). Each standby is registered with WalSndCtl when a new WALSender is started in function InitWalSenderSlot(). As 'AAA' has been registered first it becomes preferred sync standby even if priorities of both are same. When 'AAA' goes down, it marks that Slot entry as free (by setting pid=0 in function WalSndKill), now when 'AAA' comes back again, it gets that free Slot entry and again becomes preferred sync standby. So, when a user says they don't care about which standby is in sync mode, the system can decide (based on internal implementation details not visible to the user) that any node which comes online could now become the sync node, and freeze commits on the source database until that newly added node catches up? Or does the existing sync node continue in that role until the new preferred node is caught up? Existing sync node continue in that role until the new preferred node is streamed. For example, When 'AAA' comes back again and reached consistency point, 'AAA' wal sender state is 'catchup'. IWO 'AAA' doesn't has authority to operate the queue. 'BBB' server is preferred sync standby server while 'AAA' wal sender state is 'catchup'. So even if user execute query, master server doesn't freeze commits When 'AAA' walsender state become 'streaming', 'AAA' becomes preferred sync standby again. Regards, --- Sawada Masahiko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Behaviour of take over the synchronous replication
On Sat, Aug 24, 2013 at 2:46 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sat, Aug 24, 2013 at 3:14 AM, Josh Berkus j...@agliodbs.com wrote: On 08/23/2013 12:42 AM, Sawada Masahiko wrote: in case (a), those priority is clear. So I think that re-taking over is correct behaviour. OHOT, in case (b), even if AAA and BBB are set same priority, AAA server steals SYNC replication. I think it is better that BBB server continue behaviour SYNC standby, and AAA should become potential server. So, you're saying that: 1) synchronous_standby_names = '*' 2) replica 'BBB' is the current sync standby 3) replica 'AAA' comes online 4) replica 'AAA' grabs sync status ? I'm sorry that you are confuse. It means that 1) synchronous_standby_names = '*' 2) replica 'AAA' is the current sync standby 3) replica 'BBB' is the current async standby (potential sync standby) 4) replica 'AAA' fail. after that, replica 'BBB' is current sync standby. 5) replica 'AAA' comes online 6) replica 'AAA' grabs sync status If that's the case, I'm not really sure that's undesirable behavior. One could argue fairly persuasively that if you care about the precendence order of sync replicas, you shouldn't use '*'. And the rule of if using *, the lowest-sorted replica name has sync is actually a predictable, easy-to-understand rule. So if you want to make this a feature request, you'll need to come up with an argument as to why the current behavior is bad. Otherwise, you're just asking us to document it better (which is a good idea). It is not depend on name of standby server. That is, The standby server, which was connected to the master server during initial configration replication, is top priority even if priority of two server are same. What is happening here is that incase of '*' as priority of both are same, system will choose whichever comes in list of registered standby's first (list is maintained in structure WalSndCtl). Each standby is registered with WalSndCtl when a new WALSender is started in function InitWalSenderSlot(). As 'AAA' has been registered first it becomes preferred sync standby even if priorities of both are same. When 'AAA' goes down, it marks that Slot entry as free (by setting pid=0 in function WalSndKill), now when 'AAA' comes back again, it gets that free Slot entry and again becomes preferred sync standby. Now if we want to fix as you are suggesting which I don't think is necessary, we might need to change WalSndKill and some other place so that whenever any standby goes down, it changes slots for already registered standby's. User must remember that which standby server connected to master server at first. I think that this behavior confuse user. so I think that we need to modify this behaviour or if '*' is used, priority of server is not same (modifying manual is also good). Here user has done the settings (setting synchronous_standby_names = '*'), after which he will not have any control which standby will become sync standby, so ideally he should not complain. It might be case that for some users current behavior is good enough which means that with '*' whichever standby has become sync standby first, it will be the sync standby always if alive. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Behaviour of take over the synchronous replication
On Sat, Aug 24, 2013 at 3:14 AM, Josh Berkus j...@agliodbs.com wrote: On 08/23/2013 12:42 AM, Sawada Masahiko wrote: in case (a), those priority is clear. So I think that re-taking over is correct behaviour. OHOT, in case (b), even if AAA and BBB are set same priority, AAA server steals SYNC replication. I think it is better that BBB server continue behaviour SYNC standby, and AAA should become potential server. So, you're saying that: 1) synchronous_standby_names = '*' 2) replica 'BBB' is the current sync standby 3) replica 'AAA' comes online 4) replica 'AAA' grabs sync status ? I'm sorry that you are confuse. It means that 1) synchronous_standby_names = '*' 2) replica 'AAA' is the current sync standby 3) replica 'BBB' is the current async standby (potential sync standby) 4) replica 'AAA' fail. after that, replica 'BBB' is current sync standby. 5) replica 'AAA' comes online 6) replica 'AAA' grabs sync status If that's the case, I'm not really sure that's undesirable behavior. One could argue fairly persuasively that if you care about the precendence order of sync replicas, you shouldn't use '*'. And the rule of if using *, the lowest-sorted replica name has sync is actually a predictable, easy-to-understand rule. So if you want to make this a feature request, you'll need to come up with an argument as to why the current behavior is bad. Otherwise, you're just asking us to document it better (which is a good idea). It is not depend on name of standby server. That is, The standby server, which was connected to the master server during initial configration replication, is top priority even if priority of two server are same. User must remember that which standby server connected to master server at first. I think that this behavior confuse user. so I think that we need to modify this behaviour or if '*' is used, priority of server is not same (modifying manual is also good). Regards, --- Sawada Masahiko
[HACKERS] Behaviour of take over the synchronous replication
Hi all, I understand that setting synchronous_standby_name to '*' means that all priority of standby server are same. and the standby server, which connected to the master server at first, become SYNC standby, another server become ASYNC standby as potential server. So, how to set the priority to standby server is different between '*' and setting all standby names. For example, if there are two standby servers (AAA and BBB) (a) synchronous_standby_names = 'AAA,BBB' application_name | sync_priority | sync_state --+---+ AAA | 1 | sync BBB | 2 | potential (b) synchronous_standby_names = '*' application_name | sync_priority | sync_state --+---+ AAA | 1 | sync BBB | 1 | potential If the AAA server is failed, BBB server take over SYNC replication. After take over, when AAA server starts, AAA server re-take over SYNC replication from BBB server in both case (a) and case(b). That is, AAA server starts as SYNC server again. in case (a), those priority is clear. So I think that re-taking over is correct behaviour. OHOT, in case (b), even if AAA and BBB are set same priority, AAA server steals SYNC replication. I think it is better that BBB server continue behaviour SYNC standby, and AAA should become potential server. What am I missing? Please give me feedback. Regards, --- Sawada Masahiko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Behaviour of take over the synchronous replication
On 08/23/2013 12:42 AM, Sawada Masahiko wrote: in case (a), those priority is clear. So I think that re-taking over is correct behaviour. OHOT, in case (b), even if AAA and BBB are set same priority, AAA server steals SYNC replication. I think it is better that BBB server continue behaviour SYNC standby, and AAA should become potential server. So, you're saying that: 1) synchronous_standby_names = '*' 2) replica 'BBB' is the current sync standby 3) replica 'AAA' comes online 4) replica 'AAA' grabs sync status ? If that's the case, I'm not really sure that's undesirable behavior. One could argue fairly persuasively that if you care about the precendence order of sync replicas, you shouldn't use '*'. And the rule of if using *, the lowest-sorted replica name has sync is actually a predictable, easy-to-understand rule. So if you want to make this a feature request, you'll need to come up with an argument as to why the current behavior is bad. Otherwise, you're just asking us to document it better (which is a good idea). -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
