[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2019-11-21 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979442#comment-16979442 ] Michael Stack commented on HBASE-21035: --- To address more directly [~allan163]'s original

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-11-03 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673987#comment-16673987 ] Allan Yang commented on HBASE-21035: [~stack], no, it is not. The reason is that I stopped RS first,

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-11-02 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673369#comment-16673369 ] stack commented on HBASE-21035: --- Thanks [~allan163]. Yours was not the issue over in HBASE-21425? You

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-11-01 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672484#comment-16672484 ] Allan Yang commented on HBASE-21035: {quote} I tried it Allan Yang HBASE-21425 {quote} It's our

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-11-01 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672115#comment-16672115 ] stack commented on HBASE-21035: --- I tried it [~allan163] HBASE-21425 What versions did you try? > Meta

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-10-31 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671101#comment-16671101 ] stack commented on HBASE-21035: --- This has been haunting me. [~elserj] had an issue he hoisted up on to the

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-10-31 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671049#comment-16671049 ] Allan Yang commented on HBASE-21035: The problem mentioned in this issue need to be discussed again.

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621413#comment-16621413 ] Duo Zhang commented on HBASE-21035: --- Maybe a flag to indicate whether to persist? And reset it every

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-18 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619097#comment-16619097 ] stack commented on HBASE-21035: --- So, a filter on state changes and we only persist critical info? Worth

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-18 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618934#comment-16618934 ] Allan Yang commented on HBASE-21035: {quote} Then the master crashes and restarts, I think it is OK

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618590#comment-16618590 ] Duo Zhang commented on HBASE-21035: --- Theoretically there are changes, since the state has been changed

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-16 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16616991#comment-16616991 ] Allan Yang commented on HBASE-21035: Could we just do not update procedure wal if the status is not

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16616711#comment-16616711 ] Duo Zhang commented on HBASE-21035: --- OK I met the same problem with you sir [~stack]. When testing

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-14 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16615142#comment-16615142 ] stack commented on HBASE-21035: --- [~Apache9] Yes. Github. Am taking the opportunity to do some experiments.

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-14 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614558#comment-16614558 ] Duo Zhang commented on HBASE-21035: --- We could also help on HBCK2. What is the working flow for it?

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-13 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614382#comment-16614382 ] stack commented on HBASE-21035: --- I've been trying to make basic progress on HBCK2. I pushed up an HBCK2

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-13 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614231#comment-16614231 ] Allan Yang commented on HBASE-21035: {quote} So let's start helping on HBCK2? {quote} Sure! > Meta

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-13 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613263#comment-16613263 ] Duo Zhang commented on HBASE-21035: --- But for InitMetaProcedure, it could happen in normal case, if

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-12 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613009#comment-16613009 ] Allan Yang commented on HBASE-21035: Since [~stack] is running into this problem too, I still want

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-12 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16612645#comment-16612645 ] stack commented on HBASE-21035: --- I put up a patch on HBASE-21191 that steals [~allan163]'s patch from

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-12 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16612261#comment-16612261 ] stack commented on HBASE-21035: --- bq. Shouldn't we just trust the meta location in the ZK node? Maybe I

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611658#comment-16611658 ] Duo Zhang commented on HBASE-21035: --- I think the most difficult thing here is that, how do we

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-12 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611647#comment-16611647 ] Allan Yang commented on HBASE-21035: {quote} Tricky part is finding all the meta WALs and then

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-11 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611629#comment-16611629 ] stack commented on HBASE-21035: --- .002 is a hack on top of [~allan163] 's patch. I used it doing fixup on a

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-11 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611514#comment-16611514 ] Allan Yang commented on HBASE-21035: {quote} Oh, I should have mentioned, I removed all master proc

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-11 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610550#comment-16610550 ] stack commented on HBASE-21035: --- Oh, I should have mentioned, I removed all master proc WALs at one stage

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-11 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610534#comment-16610534 ] stack commented on HBASE-21035: --- bq. I think Allan Yang has a valid point. I suppose they have not

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-11 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610520#comment-16610520 ] Duo Zhang commented on HBASE-21035: --- I think [~allan163] has a valid point. For SCP, if meta is not

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-11 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610503#comment-16610503 ] stack commented on HBASE-21035: --- bq. stack, sir, I don't get it, how can SCP finish without meta online?

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-11 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610238#comment-16610238 ] Allan Yang commented on HBASE-21035: {quote} i.e. meta is not online even though the above server's

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-08 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608297#comment-16608297 ] Duo Zhang commented on HBASE-21035: --- I think one thing we could do is that, collect the WAL directory

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-08 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608285#comment-16608285 ] stack commented on HBASE-21035: --- Let me do what you suggest first. > Meta Table should be able to online

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-08 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608254#comment-16608254 ] Duo Zhang commented on HBASE-21035: --- If the cluster is not in a good state and several RSes keep

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-08 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608185#comment-16608185 ] stack commented on HBASE-21035: --- I'm back. My master is aborting after spending fours reconstructing the

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-09-05 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605123#comment-16605123 ] stack commented on HBASE-21035: --- Late to the party I've been playing with removing the procedure WALs

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576279#comment-16576279 ] Duo Zhang commented on HBASE-21035: --- The goal for HBASE-20708 is to remove the unnecessary scheduling

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576240#comment-16576240 ] Allan Yang commented on HBASE-21035: If scheduling a SCP for servers with '-splitting' is not a good

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576233#comment-16576233 ] Hadoop QA commented on HBASE-21035: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576229#comment-16576229 ] Duo Zhang commented on HBASE-21035: --- What if you lose some edits of meta region and then bring the

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576204#comment-16576204 ] Allan Yang commented on HBASE-21035: Or in short, my opinion is that, we have to try our best to

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576189#comment-16576189 ] Allan Yang commented on HBASE-21035: {quote} You bring meta online and mess up all the data and

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576096#comment-16576096 ] Duo Zhang commented on HBASE-21035: --- Anyway I'm -1 on doing anything automatically to try to recover

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Allan Yang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576089#comment-16576089 ] Allan Yang commented on HBASE-21035: {quote} I think this should a tool in HBCK? As in the normal

[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost

2018-08-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576074#comment-16576074 ] Duo Zhang commented on HBASE-21035: --- I think this should a tool in HBCK? As in the normal code path,