[
https://issues.apache.org/jira/browse/HBASE-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145059#comment-13145059
]
Phabricator commented on HBASE-4742:
------------------------------------
khemani has commented on the revision "[jira] [HBASE-4742] Split dead server's
log in parallel".
If you splitLog(list-of-servernames) then the function won't return until all
the servers' logs are split.
If you call multiple splitLog(servername) then each one returns as soon as it
is done. This is what we are doing now and we should stay with it.
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/master/HMaster.java:305 managing
threads through a thread pool is a good thing. But I doubt that there is any
scenario where the number of threads will increase without limit. The number of
threads will always be limited by the number of region servers. And a new
incarnation of a region server is not instantiated by the master unless the old
one's log splitting is done.
but i think we should still use the thread pool. it is a better abstraction.
src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java:304
Can you please do a new Runnable inline over here. That will make it easier for
someone to read ProcessServerShutdown() code. Having two methods run() and
process() kind of makes it confusing to understand ProcessServerShutdown.
src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java:323
This should be LOG.debug. You might consider rate limiting this debug output.
src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java:333
This should be LOG.warn
src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java:339
Runtime.halt()? Even if you throw an exception the retry mechanism will kick in.
src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java:304
If master shutdown is called (abort or regular shutdown) then I am not sure
that the pool is shutting down properly. Someone has to interrupt each of the
threads that are have called splitLog(). Or those splitLog() threads have to be
daemons.
src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java:316
Now that you have a thread pool you can look at the status of the submitted
tasks instead of maintaining special states :)
REVISION DETAIL
https://reviews.facebook.net/D237
> Split dead server's log in parallel
> -----------------------------------
>
> Key: HBASE-4742
> URL: https://issues.apache.org/jira/browse/HBASE-4742
> Project: HBase
> Issue Type: Improvement
> Reporter: Liyin Tang
> Assignee: Liyin Tang
> Attachments: D237.1.patch, D237.2.patch, D237.3.patch, D237.4.patch,
> D237.5.patch
>
>
> When one region server goes down, the master will shutdown the region server
> and split its log.
> However, splitting log is a blocking call and it would take some time.
> If more than one region server go down, the master will split its log one by
> one, which is not efficient.
> Since we have the distributed log split, we could split these logs from the
> dead servers in parallel.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
