[
https://issues.apache.org/jira/browse/HBASE-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003648#comment-15003648
]
Ashu Pachauri commented on HBASE-14802:
---------------------------------------
[~stack] I actually encountered one of its side effects i.e. the balancer fails
to run in such a scenario. When I debug my way through this, I found this
issue. I did not try, but this should be reproducible in Integration tests too,
by expiring a few RS at once and failing over the master shortly after that.
[~stack] [~mbertozzi] I also had the same concern when I started working on
this; there might be a more elegant solution to this problem. I am also not
very familiar with this bit of code, but it seems that we need DeadServer to be
aware of the fact that a separate procedure is processing the recovery which
can be replayed many times over; which should be possible in other ways than
leaking procID.
> Replaying server crash recovery procedure after a failover causes incorrect
> handling of deadservers
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-14802
> URL: https://issues.apache.org/jira/browse/HBASE-14802
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 2.0.0, 1.2.0, 1.2.1
> Reporter: Ashu Pachauri
> Assignee: Ashu Pachauri
> Attachments: HBASE-14802.patch
>
>
> The way dead servers are processed is that a ServerCrashProcedure is launched
> for a server after it is added to the dead servers list.
> Every time a server is added to the dead list, a counter "numProcessing" is
> incremented and it is decremented when a crash recovery procedure finishes.
> Since, adding a dead server and recovering it are two separate events, it can
> cause inconsistencies.
> If a master failover occurs in the middle of the crash recovery, the
> numProcessing counter resets but the ServerCrashProcedure is replayed by the
> new master. This causes the counter to go negative and makes the master think
> that dead servers are still in process of recovery.
> This has ramifications on the balancer that the balancer ceases to run after
> such a failover.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
