[jira] [Commented] (HBASE-5528) Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted
[
https://issues.apache.org/jira/browse/HBASE-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223475#comment-13223475
]
Prakash Khemani commented on HBASE-5528:
I think the log-splitting retry logic is there in ServerShutdownHandler ...
In ServerShutdownHandler.process() ... the handler is requeued in case of error
code
try {
if (this.shouldSplitHlog) {
LOG.info(Splitting logs for + serverName);
this.services.getMasterFileSystem().splitLog(serverName);
} else {
LOG.info(Skipping log splitting for + serverName);
}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
code
Retry splitting log if failed in the process of ServerShutdownHandler, and
abort master when retries exhausted
--
Key: HBASE-5528
URL: https://issues.apache.org/jira/browse/HBASE-5528
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: hbase-5528.patch, hbase-5528v2.patch
We will retry splitting log if failed in splitLogAfterStartup when master
starts.
However, there is no retry for failed splitting log in the process of
ServerShutdownHandler.
Also, if we finally failed to split log, we should abort master even if
filesystem is ok to prevent data loss.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5528) Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted
[ https://issues.apache.org/jira/browse/HBASE-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223911#comment-13223911 ] chunhui shen commented on HBASE-5528: - @Prakash I'm sorry I haven't found that I'm using 90 version. However, in current log-splitting retry logic, it will retry forever if throws IOException, I think it's unacceptable. Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted -- Key: HBASE-5528 URL: https://issues.apache.org/jira/browse/HBASE-5528 Project: HBase Issue Type: Bug Reporter: chunhui shen Assignee: chunhui shen Attachments: hbase-5528.patch, hbase-5528v2.patch We will retry splitting log if failed in splitLogAfterStartup when master starts. However, there is no retry for failed splitting log in the process of ServerShutdownHandler. Also, if we finally failed to split log, we should abort master even if filesystem is ok to prevent data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5528) Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted
[
https://issues.apache.org/jira/browse/HBASE-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223001#comment-13223001
]
Zhihong Yu commented on HBASE-5528:
---
{code}
+if (!checkFileSystem()) {
+ LOG.warn(Bad Filesystem, exiting);
+ retrySplittingNum = 0;
+} else {
{code}
The above code changes the way we respond to filesystem check failure where
Runtime.getRuntime().halt(1) should be called.
Retry splitting log if failed in the process of ServerShutdownHandler, and
abort master when retries exhausted
--
Key: HBASE-5528
URL: https://issues.apache.org/jira/browse/HBASE-5528
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: hbase-5528.patch
We will retry splitting log if failed in splitLogAfterStartup when master
starts.
However, there is no retry for failed splitting log in the process of
ServerShutdownHandler.
Also, if we finally failed to split log, we should abort master even if
filesystem is ok to prevent data loss.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5528) Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted
[ https://issues.apache.org/jira/browse/HBASE-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223021#comment-13223021 ] Lars Hofhansl commented on HBASE-5528: -- This seems like an important fix. Could you upload to review board for easier review? Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted -- Key: HBASE-5528 URL: https://issues.apache.org/jira/browse/HBASE-5528 Project: HBase Issue Type: Bug Reporter: chunhui shen Assignee: chunhui shen Attachments: hbase-5528.patch, hbase-5528v2.patch We will retry splitting log if failed in splitLogAfterStartup when master starts. However, there is no retry for failed splitting log in the process of ServerShutdownHandler. Also, if we finally failed to split log, we should abort master even if filesystem is ok to prevent data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5528) Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted
[ https://issues.apache.org/jira/browse/HBASE-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223030#comment-13223030 ] chunhui shen commented on HBASE-5528: - https://reviews.apache.org/r/4194/ Retry splitting log if failed in the process of ServerShutdownHandler, and abort master when retries exhausted -- Key: HBASE-5528 URL: https://issues.apache.org/jira/browse/HBASE-5528 Project: HBase Issue Type: Bug Reporter: chunhui shen Assignee: chunhui shen Attachments: hbase-5528.patch, hbase-5528v2.patch We will retry splitting log if failed in splitLogAfterStartup when master starts. However, there is no retry for failed splitting log in the process of ServerShutdownHandler. Also, if we finally failed to split log, we should abort master even if filesystem is ok to prevent data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
