[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195103#comment-16195103 ] Hudson commented on HBASE-14004: Results for branch HBASE-18467, done in 4 hr 24 min and counting

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185802#comment-16185802 ] Hudson commented on HBASE-14004: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3798 (See

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185726#comment-16185726 ] Hudson commented on HBASE-14004: FAILURE: Integrated in Jenkins build HBase-2.0 #597 (See

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185257#comment-16185257 ] Duo Zhang commented on HBASE-14004: --- Will do. Thanks for the reminding sir. > [Replication]

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184854#comment-16184854 ] stack commented on HBASE-14004: --- [~Apache9] Any chance of a release note here please boss? This is a great

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170982#comment-16170982 ] Duo Zhang commented on HBASE-14004: --- OK, will check later. > [Replication] Inconsistency between

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170374#comment-16170374 ] Ted Yu commented on HBASE-14004: TestReplicationSmallTests fails consistently on master. If I switch to

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169643#comment-16169643 ] Duo Zhang commented on HBASE-14004: --- The problem for FSHLog is complicated. One thing is that, syncing

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169589#comment-16169589 ] Sean Busbey commented on HBASE-14004: - is this needed in branch-1? the doc mentions that the

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168299#comment-16168299 ] stack commented on HBASE-14004: --- This is a great fix. > [Replication] Inconsistency between Memstore and

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168034#comment-16168034 ] Hudson commented on HBASE-14004: FAILURE: Integrated in Jenkins build HBase-2.0 #518 (See

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167998#comment-16167998 ] Hudson commented on HBASE-14004: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3719 (See

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167727#comment-16167727 ] Duo Zhang commented on HBASE-14004: --- OK, all green. Let me commit. > [Replication] Inconsistency

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167720#comment-16167720 ] Hadoop QA commented on HBASE-14004: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167437#comment-16167437 ] Hadoop QA commented on HBASE-14004: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165857#comment-16165857 ] stack commented on HBASE-14004: --- [~Apache9] Sorry. I reviewed it a few days ago but forgot to publish the

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165819#comment-16165819 ] Duo Zhang commented on HBASE-14004: --- Thanks [~ram_krish] for reviewing. Will commit later. >

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164323#comment-16164323 ] ramkrishna.s.vasudevan commented on HBASE-14004: Will take a look at this @duo zhang. >

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164280#comment-16164280 ] Duo Zhang commented on HBASE-14004: --- Ping [~stack] [~anoopsamjohn] [~ram_krish], this feature is

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162524#comment-16162524 ] Hadoop QA commented on HBASE-14004: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162439#comment-16162439 ] Duo Zhang commented on HBASE-14004: --- [~stack] [~anoopsamjohn] PTAL. Thanks. > [Replication]

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162335#comment-16162335 ] Duo Zhang commented on HBASE-14004: --- Good, no test failures. Let me fix the javadoc issues and upload

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161489#comment-16161489 ] Hadoop QA commented on HBASE-14004: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15625326#comment-15625326 ] Duo Zhang commented on HBASE-14004: --- I think we need to pick this up. With AsyncFSWAL, it is not safe

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149552#comment-15149552 ] stack commented on HBASE-14004: --- But, as noted elsewhere, this fact does not solve this issue (what made it

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149497#comment-15149497 ] stack commented on HBASE-14004: --- bq. We could try rejiggering the order in which memstore gets updated,

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050322#comment-15050322 ] Yu Li commented on HBASE-14004: --- bq. Yu, is it still a problem after we not using hsync? My concern was

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050409#comment-15050409 ] Yu Li commented on HBASE-14004: --- Let me confirm, do you mean we never rollback the memstore even if

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050418#comment-15050418 ] Heng Chen commented on HBASE-14004: --- {quote} Let me confirm, do you mean we never rollback the memstore

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050467#comment-15050467 ] Phil Yang commented on HBASE-14004: --- We can not rollback even if buffered-entries rewrite failed, or we

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050308#comment-15050308 ] Phil Yang commented on HBASE-14004: --- [~chenheng] it seems that HBASE-14949 only handle replaying logs

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050333#comment-15050333 ] Heng Chen commented on HBASE-14004: --- That is my plan. > [Replication] Inconsistency between Memstore

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050332#comment-15050332 ] Heng Chen commented on HBASE-14004: --- That is my plan. > [Replication] Inconsistency between Memstore

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050376#comment-15050376 ] Heng Chen commented on HBASE-14004: --- {quote} 2. WAL logger tries to re-write the buffered entries to a

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050438#comment-15050438 ] Heng Chen commented on HBASE-14004: --- IMO we should handle this situation, if we failed during recovery

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048552#comment-15048552 ] Phil Yang commented on HBASE-14004: --- You are right, we should change the logic of replicator. And I am

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049048#comment-15049048 ] Phil Yang commented on HBASE-14004: --- The reason I asked this question is if we don't have to make sure

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049947#comment-15049947 ] Heng Chen commented on HBASE-14004: --- {quote} 1, 2, 3, 4, 5---this is normal order 1, 3, 2, 4, 5---the

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050165#comment-15050165 ] Phil Yang commented on HBASE-14004: --- I have revised the doc

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049009#comment-15049009 ] stack commented on HBASE-14004: --- Not sure I understand the question. Replication reads the WAL in order and

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046584#comment-15046584 ] Heng Chen commented on HBASE-14004: --- I make a patch for skip duplicate entries when replay, i found

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046656#comment-15046656 ] stack commented on HBASE-14004: --- Doc is great. I added a few little notes... > [Replication] Inconsistency

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047929#comment-15047929 ] Heng Chen commented on HBASE-14004: --- {quote} Have to be careful how we do this. We have elaborate

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047986#comment-15047986 ] Heng Chen commented on HBASE-14004: --- {quote} But I think there should be a follow-up work on data

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047976#comment-15047976 ] Phil Yang commented on HBASE-14004: --- {quote} And as our design, we only use hsync to ensure data

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048171#comment-15048171 ] Heng Chen commented on HBASE-14004: --- {quote} And if we don't use hsync, I think we need not change the

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048137#comment-15048137 ] Phil Yang commented on HBASE-14004: --- {quote} And in this issue, i suppose we just ensure correctness

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048010#comment-15048010 ] stack commented on HBASE-14004: --- This is the issue where client can say what durability to apply per edit.

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046414#comment-15046414 ] Heng Chen commented on HBASE-14004: --- Why we need 'where to end' when recover replication? We just

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046475#comment-15046475 ] Yu Li commented on HBASE-14004: --- You are right Heng, and Duo also explained about this. Thanks. :-) >

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046416#comment-15046416 ] Heng Chen commented on HBASE-14004: --- Why we need 'where to end' when recover replication? We just

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046415#comment-15046415 ] Heng Chen commented on HBASE-14004: --- Why we need 'where to end' when recover replication? We just

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046422#comment-15046422 ] Duo Zhang commented on HBASE-14004: --- I do not think we can benefit a lot from storing acked length to

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046439#comment-15046439 ] Yu Li commented on HBASE-14004: --- bq. HBase uses NN to do fencing which means HMaster will only consider an

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044970#comment-15044970 ] Phil Yang commented on HBASE-14004: --- Furthermore, I find another issue that since HDFS-744 supported

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044563#comment-15044563 ] Heng Chen commented on HBASE-14004: --- Looks good. Except one thing. {code} If we get timeout error

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044643#comment-15044643 ] Phil Yang commented on HBASE-14004: --- {quote} It seems that WAL reader is designed for one WAL, we can't

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044685#comment-15044685 ] Phil Yang commented on HBASE-14004: --- {quote} Guess this may be the original design since there is TODO

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044576#comment-15044576 ] Phil Yang commented on HBASE-14004: --- {quote} Some entries in new WAL may have smaller seqid than

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044616#comment-15044616 ] He Liangliang commented on HBASE-14004: --- Sorry for the late reply. What about this approach: 1.

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044666#comment-15044666 ] Heng Chen commented on HBASE-14004: --- {quote} You are right, the logic that ignores logs which are

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044603#comment-15044603 ] Heng Chen commented on HBASE-14004: --- It seems that WAL reader is designed for one WAL, we can't

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044702#comment-15044702 ] He Liangliang commented on HBASE-14004: --- We don't need to guarantee "avoid replaying or replicating

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044835#comment-15044835 ] Phil Yang commented on HBASE-14004: --- {quote} In another word, the client needn't discriminate between

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046226#comment-15046226 ] Yu Li commented on HBASE-14004: --- bq. My idea may be more fast because it needn't send request to zk

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046286#comment-15046286 ] Phil Yang commented on HBASE-14004: --- {quote} I guess we need to persist the acked length to somewhere

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046292#comment-15046292 ] Heng Chen commented on HBASE-14004: --- {quote} I guess we need to persist the acked length to somewhere

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046371#comment-15046371 ] Yu Li commented on HBASE-14004: --- It seems to me that replication queue only records where to *start*

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046385#comment-15046385 ] Yu Li commented on HBASE-14004: --- Assume this situation: 1. RS received an ack from DFSClient, and record it

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046395#comment-15046395 ] Duo Zhang commented on HBASE-14004: --- {quote} If e don't persist this acked length, during failover we

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046398#comment-15046398 ] Duo Zhang commented on HBASE-14004: --- {quote} 3. Replication queue recovery starts on another RS, where

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046401#comment-15046401 ] Yu Li commented on HBASE-14004: --- bq. then the file will finally be closed by NN I don't think *finally be

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046405#comment-15046405 ] Duo Zhang commented on HBASE-14004: --- {quote} I don't think finally be closed is enough, since the

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044318#comment-15044318 ] Heng Chen commented on HBASE-14004: --- {quote} If only RS crashed, DNs not, there is no data loss because

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044319#comment-15044319 ] Duo Zhang commented on HBASE-14004: --- {quote} Is it means some entries may be already in place (you

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044321#comment-15044321 ] Duo Zhang commented on HBASE-14004: --- {quote} If one hlush failed, we close old WAL and write entries

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044308#comment-15044308 ] Heng Chen commented on HBASE-14004: --- {quote} To be clear, this means we will hold the WAL.sync request

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044315#comment-15044315 ] Phil Yang commented on HBASE-14004: --- {quote} If RS crashed at this time, what will happen? {quote} If

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044327#comment-15044327 ] Heng Chen commented on HBASE-14004: --- {quote} No. You can close old WAL using any length that larger

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044341#comment-15044341 ] Heng Chen commented on HBASE-14004: --- {quote} Now HBase only use hflush, if we add a hsync logic, will

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044380#comment-15044380 ] Phil Yang commented on HBASE-14004: --- Sure, I'm going to draft a document for summarizing our ideas.

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044368#comment-15044368 ] Heng Chen commented on HBASE-14004: --- {quote} btw, could see quite some discussion here and maybe a doc

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044333#comment-15044333 ] Phil Yang commented on HBASE-14004: --- I have a new concern about the configuration. Now HBase only use

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044355#comment-15044355 ] Yu Li commented on HBASE-14004: --- bq. if we add a hsync logic, will there be degradation of performance IMHO

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043721#comment-15043721 ] Yu Li commented on HBASE-14004: --- Nice discussion [~yangzhe1991] and [~Apache9]. FWIW, two questions about

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043246#comment-15043246 ] Duo Zhang commented on HBASE-14004: --- Sounds great, an incremental unique id of WAL entry is better

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042765#comment-15042765 ] Phil Yang commented on HBASE-14004: --- Is it required to use the size of serialized binary data? I don't

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042744#comment-15042744 ] Duo Zhang commented on HBASE-14004: --- {quote} Fixing HBase writing path that we should retry logging WAL

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043657#comment-15043657 ] stack commented on HBASE-14004: --- bq. ReplicationSource should only read WAL that is hsynced to prevent

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043664#comment-15043664 ] Duo Zhang commented on HBASE-14004: --- {quote} This will require big change in how replication works but

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042694#comment-15042694 ] Phil Yang commented on HBASE-14004: --- After discussion in HBASE-14790 , we can move forward now. Let me

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999866#comment-14999866 ] Yu Li commented on HBASE-14004: --- [~chenheng] [~Apache9] Sorry for the late response, somehow didn't notice

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977908#comment-14977908 ] Anoop Sam John commented on HBASE-14004: In case when RS is killed and so later WAL replay will

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977841#comment-14977841 ] Heng Chen commented on HBASE-14004: --- {quote} What if the regionserver crashed before flushing HFile? I

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978108#comment-14978108 ] Duo Zhang commented on HBASE-14004: --- [~carp84] This is an inherent problem of RPC based systems due to

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978004#comment-14978004 ] Yu Li commented on HBASE-14004: --- {quote} It's possible that the HDFS sync RPC call fails, but the data is

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978038#comment-14978038 ] Heng Chen commented on HBASE-14004: --- On the assumption that HDFS sync successfully and do response,

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977691#comment-14977691 ] Heng Chen commented on HBASE-14004: --- I have a proposal. When WAL sync failed, and master has to

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977696#comment-14977696 ] Heng Chen commented on HBASE-14004: --- {quote} When WAL sync failed, and master has to rollback Memstore.

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977769#comment-14977769 ] Duo Zhang commented on HBASE-14004: --- The problem here not only effects replication. {quote} As a

  1   2   >