I do not remember all the details but I believe the recovery lease issue had to do with jimmy and the locality improvements was likely referring to Deveraj's work.
Jon. On Mon, May 6, 2013 at 3:21 PM, Ted Yu <[email protected]> wrote: > Jon: > Thanks for the notes. > Do you remember a bit more detail for the following two topics ? > > 'recover lease thing' and 'locality improvements' > > Cheers > > On Mon, May 6, 2013 at 11:43 AM, Jonathan Hsieh <[email protected]> wrote: > > > These are my rough notes from a meatspace meeting was held on 5/3/15 > > > > tl;dr a large group of folks pushing on the 0.95 release went through the > > myriad of jirae and hashed out which ones we should try to get into the > > next 0.95 release, and bumped several issues from the 0.95.1 release. > > > > There was also a substantial discussion about getting hadoop2 tests to > pass > > (and understanding why the currently fail) and also about the new table > > namespacing feature. > > > > Other topics: more people please review, BoF session for Hadoop Summit, > > Integration testing suite. > > > > ---- > > > > (notes get sparser towards the end). > > > > 5/3/13 > > Location: Hortonworks HQ, Palo Alto, CA > > Who: Stack, JD, Jon H , Jimmy X, Himanshu, Dave W, Elliott C, Ted Y, > Enis, > > Deveraj, Sergey, Jeff Z, (Matteo, Nicholas L on the phone). > > > > Went through blockers and critical lists (goal, resolve or nix before > > release): > > > > Blocker: > > HBASE-5995 - logrol on dn pipeline restart (in progress) > > > > Critical: > > HBASE-8366 logs full trace stuff. > > HBASE-5923 check and clean - nix > > HBASE-8450 - hbase-defaults.xml updates - stack to do, everyone please > look > > at it > > HBASE-7997 - on last class moves. - not critical > > HBASE-6891 - hadoop2 - investigate speed regression is hadoop2 or > hadoop1. > > * make unit test pass right new. regression investigation not blocking. > > * HBASE-8337 Hadoop2 - SCR problem - can resolve. have reasonable > reason > > HBASE-7006 - mttr - distributed log replay - give feature good name and > > close it out. > > HBASE-3787 - increment non-idepmottent - sergey - working on tests. > > HBASE-8449 - recover lease "fun and games" - stack working on it. > > HBASE-7932 - region locations in meta. > > HBASE-7897 - cells and tags. change interface. cell interface must be > > hardened in 0.96. - big for intel > > HBASE-8483 - zk leak. > > > > Selected Majors to bump to critical?: > > > > > > HBASE-8143 - short circuit read. OOM. every reader 1MB buffer, 600 > block > > readers, then lots of memory used. > > - move to critical - enis: maybe set to hfile block size > > > > Would be good to fix: > > HBASE-8385 - > > HBASE-7910 - > > HBASE-7391 - > > HBASE-7709 - infite loop in m/m replication - would be good to fix. > > HBASE-7564 - replication refactor. -- if done done, if not leave out. > > HBASE-7958 - stats per column family per-region - remove from 0.95.1 > > HBASE-6294 - usability, nice to fix. > > HBASE-8479 - compile issue - generics problem. > > HBASE-4050 - metrics to metric2 umbrella blocked by HBASE-7074 - docs for > > hbase-4050 > > HBASE-6580 htable pool broken - leaving in for now. > > HBASE-7839 - dead machine in integration - nice to have > > HBASE-7840 - nice to have > > > > ---- > > Enis: > > > > HBASE-8015 namespaces (discussion below) > > - file system structure likely changes things, ideally before 0.6 > > HBASE-6721 RS grouping stuff- can do with coproc and load balancer > plugins > > (not blocking0. > > HBASE-7999 system tables? (deprecated if namespaces added) > > > > namespace descriptor - > > - distribute in cluster. similar to zk permisison watcher > > > > '.' vs some other separator char for <namespace><sep><table> > > - '.' is what db's use, but is valid hbase table name and breaks hbase > > tables > > - other chars would look weird but don't break existing hbase tables. > > > > the proposed dir structure move has impact on upgrades > > - do a shutdown restart upgrade? > > - Discussed potential approach where empty name space files don't move > (and > > don't break rolling upgrade) > > - discussed how to deal with matteo's hfile pool approach (one dir with > > hfiles, meta just has pointers, not dependent on dir strcuture) > > > > how does this interact with rolling upgrades? > > how does this interact with hdfs quotas? > > > > stack: > > - not in 0.95 unless progress is made. > > - Let's try to get it committed to a branch ala snapshots > > > > ---- > > > > patch available - discussion > > - please do more reviews. > > > > ---- > > > > integration tests > > - alex 94 > > - elliot 95 > > > > run via maven against distributed cluster > > - system test framework > > - cherry-pick, run via command line tool. > > - 0.94 chaos monkey - every 1 in 5 fails > > - 0.96 - 1 in 10 fails. > > > > machines don't come backup. > > > > 4 tests are good > > - test too many regions > > - want tests to scale to scale > > - want to kill less -- not completely random. > > > > bigtop does this > > - put in maven. > > > > ----- > > > > recover lease thing > > > > ---- > > > > locality improvements > > > > ---- > > > > Hadoop summit > > - day before - birds of a feather > > > > ---- > > > > compactions > > > > -- > > // Jonathan Hsieh (shay) > > // Software Engineer, Cloudera > > // [email protected] > > > -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // [email protected]
