[ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264691#comment-13264691 ]
jirapos...@reviews.apache.org commented on HBASE-5712: ------------------------------------------------------ bq. On 2012-04-27 23:27:20, Michael Stack wrote: bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 204 bq. > <https://reviews.apache.org/r/4883/diff/1/?file=104442#file104442line204> bq. > bq. > This'll work but why not ConcurrentSkipListMap? Sure, changed. bq. On 2012-04-27 23:27:20, Michael Stack wrote: bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 418 bq. > <https://reviews.apache.org/r/4883/diff/1/?file=104442#file104442line418> bq. > bq. > +1 on suggested change done. bq. On 2012-04-27 23:27:20, Michael Stack wrote: bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 2835 bq. > <https://reviews.apache.org/r/4883/diff/1/?file=104442#file104442line2835> bq. > bq. > Is this flag needed? Why not just check thread is alive? I see we can return with an error. What happens if the return on 2816 happens? Will the wait at #643 above be for ever? This is not a thread but actually fed to an executor (thread pool) at line 637. If the return happens on 2816, this is in a finally which will always mark the workitem as done. There are two other instances of this pattern that were originally in this code before I got to it -- I'd have used Futures (and have filed a follow on issue for it) but it works. - jmhsieh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4883/#review7337 ----------------------------------------------------------- On 2012-04-26 01:42:01, jmhsieh wrote: bq. bq. ----------------------------------------------------------- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/4883/ bq. ----------------------------------------------------------- bq. bq. (Updated 2012-04-26 01:42:01) bq. bq. bq. Review request for hbase, Ted Yu and Jimmy Xiang. bq. bq. bq. Summary bq. ------- bq. bq. * Parallelized load of .regioninfo files bq. * changed TreeMap to SortedMap in method signatures bq. * renamed a test's name. bq. bq. bq. This addresses bug HBASE-5712. bq. https://issues.apache.org/jira/browse/HBASE-5712 bq. bq. bq. Diffs bq. ----- bq. bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 66156c2 bq. src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java 6b64f10 bq. bq. Diff: https://reviews.apache.org/r/4883/diff bq. bq. bq. Testing bq. ------- bq. bq. Ran patch 10x on trunk, passes. Ran 1x on 0.92 and 0.94. bq. bq. Ther 0.90 version that is nearly identical except for ignoring changes near lines HBaseFsck lines 671-680. bq. bq. bq. Thanks, bq. bq. jmhsieh bq. bq. > Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. > --------------------------------------------------------------------------- > > Key: HBASE-5712 > URL: https://issues.apache.org/jira/browse/HBASE-5712 > Project: HBase > Issue Type: Improvement > Components: hbck > Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > Reporter: Jonathan Hsieh > Assignee: Jonathan Hsieh > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, > hbase-5712-v2.patch, hbase-5712.patch > > > On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off > for 60s before attempting to read data from another datanode. Portions of > the information gathered from hdfs (.regioninfo files) are loaded serially. > With HBase with clusters with 100's, or 1000's, or 10000's regions > encountering these 60s delay blocks progress and can be very painful. > There is already some parallelization of portions of the hdfs information > load operations and the goal here is move the reading of .regioninfos into > the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira