[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618694#comment-13618694 ] Brian Fu commented on HBASE-8104: - [~ctrezzo], one cluster fail refer to hbase and zookeeper cluster are unable, but HDFS cluster is ok. this funciton not like replicaiton . HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618963#comment-13618963 ] Chris Trezzo commented on HBASE-8104: - Ah I see. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617147#comment-13617147 ] Brian Fu commented on HBASE-8104: - [~ctrezzo] no. I want to implement a function that automatically write the inconsistent data to the target cluster after one cluster fail. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617591#comment-13617591 ] Chris Trezzo commented on HBASE-8104: - [~brianhbase] When you say one cluster fail what type of failure are you referring to? Are you trying to cover the case where the HBase application layer is unable to replicate data, but the HDFS cluster and ZooKeeper cluster are still available? In the other cases I can think of, your implemented function would be in the same situation as the HBase replication code (unless I am miss-understanding the scenario). HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608509#comment-13608509 ] Jieshan Bean commented on HBASE-8104: - I think so, this is the only way we can do that. But I don't think we really need that. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608541#comment-13608541 ] Enis Soztutar commented on HBASE-8104: -- In your case, you have to ensure two conditions: (1) Only one master cluster is active at any given time. (2) Data is not read from one cluster, unless all the data is replicated from the other cluster. Achieving (1) is easy. (2) might not even be possible if you do not have synchronous replication, since eventual updates might not be propagated for extended periods of time. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605360#comment-13605360 ] Chris Trezzo commented on HBASE-8104: - [~brianhbase] is your intent to make replication synchronous across HBase clusters? HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604849#comment-13604849 ] Brian Fu commented on HBASE-8104: - Thank you for your answer. Even if use and Master-Master replication, still may appear inconsistent data, Suppose we have A, B two master cluster. if A cluster have problems, so all write and read request switching to B cluster. there may be data in A cluster is not timely replication to B cluster. So some data cannot be read in B cluster timely. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13603171#comment-13603171 ] Brian Fu commented on HBASE-8104: - Hi , Lars Hofhansl Master-Master cluster replication will lead to the data endless loop in the two cluster. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13603469#comment-13603469 ] Lars Hofhansl commented on HBASE-8104: -- In 0.94+ it won't. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13602365#comment-13602365 ] Lars Hofhansl commented on HBASE-8104: -- Can you setup the two clusters in Master-Master replication? That way all changes made the slave cluster during the failover are scheduled to be replicated back to the main cluster once that becomes available. HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
