Jon: Thanks for the notes. Do you remember a bit more detail for the following two topics ?
'recover lease thing' and 'locality improvements' Cheers On Mon, May 6, 2013 at 11:43 AM, Jonathan Hsieh <[email protected]> wrote: > These are my rough notes from a meatspace meeting was held on 5/3/15 > > tl;dr a large group of folks pushing on the 0.95 release went through the > myriad of jirae and hashed out which ones we should try to get into the > next 0.95 release, and bumped several issues from the 0.95.1 release. > > There was also a substantial discussion about getting hadoop2 tests to pass > (and understanding why the currently fail) and also about the new table > namespacing feature. > > Other topics: more people please review, BoF session for Hadoop Summit, > Integration testing suite. > > ---- > > (notes get sparser towards the end). > > 5/3/13 > Location: Hortonworks HQ, Palo Alto, CA > Who: Stack, JD, Jon H , Jimmy X, Himanshu, Dave W, Elliott C, Ted Y, Enis, > Deveraj, Sergey, Jeff Z, (Matteo, Nicholas L on the phone). > > Went through blockers and critical lists (goal, resolve or nix before > release): > > Blocker: > HBASE-5995 - logrol on dn pipeline restart (in progress) > > Critical: > HBASE-8366 logs full trace stuff. > HBASE-5923 check and clean - nix > HBASE-8450 - hbase-defaults.xml updates - stack to do, everyone please look > at it > HBASE-7997 - on last class moves. - not critical > HBASE-6891 - hadoop2 - investigate speed regression is hadoop2 or hadoop1. > * make unit test pass right new. regression investigation not blocking. > * HBASE-8337 Hadoop2 - SCR problem - can resolve. have reasonable reason > HBASE-7006 - mttr - distributed log replay - give feature good name and > close it out. > HBASE-3787 - increment non-idepmottent - sergey - working on tests. > HBASE-8449 - recover lease "fun and games" - stack working on it. > HBASE-7932 - region locations in meta. > HBASE-7897 - cells and tags. change interface. cell interface must be > hardened in 0.96. - big for intel > HBASE-8483 - zk leak. > > Selected Majors to bump to critical?: > > > HBASE-8143 - short circuit read. OOM. every reader 1MB buffer, 600 block > readers, then lots of memory used. > - move to critical - enis: maybe set to hfile block size > > Would be good to fix: > HBASE-8385 - > HBASE-7910 - > HBASE-7391 - > HBASE-7709 - infite loop in m/m replication - would be good to fix. > HBASE-7564 - replication refactor. -- if done done, if not leave out. > HBASE-7958 - stats per column family per-region - remove from 0.95.1 > HBASE-6294 - usability, nice to fix. > HBASE-8479 - compile issue - generics problem. > HBASE-4050 - metrics to metric2 umbrella blocked by HBASE-7074 - docs for > hbase-4050 > HBASE-6580 htable pool broken - leaving in for now. > HBASE-7839 - dead machine in integration - nice to have > HBASE-7840 - nice to have > > ---- > Enis: > > HBASE-8015 namespaces (discussion below) > - file system structure likely changes things, ideally before 0.6 > HBASE-6721 RS grouping stuff- can do with coproc and load balancer plugins > (not blocking0. > HBASE-7999 system tables? (deprecated if namespaces added) > > namespace descriptor - > - distribute in cluster. similar to zk permisison watcher > > '.' vs some other separator char for <namespace><sep><table> > - '.' is what db's use, but is valid hbase table name and breaks hbase > tables > - other chars would look weird but don't break existing hbase tables. > > the proposed dir structure move has impact on upgrades > - do a shutdown restart upgrade? > - Discussed potential approach where empty name space files don't move (and > don't break rolling upgrade) > - discussed how to deal with matteo's hfile pool approach (one dir with > hfiles, meta just has pointers, not dependent on dir strcuture) > > how does this interact with rolling upgrades? > how does this interact with hdfs quotas? > > stack: > - not in 0.95 unless progress is made. > - Let's try to get it committed to a branch ala snapshots > > ---- > > patch available - discussion > - please do more reviews. > > ---- > > integration tests > - alex 94 > - elliot 95 > > run via maven against distributed cluster > - system test framework > - cherry-pick, run via command line tool. > - 0.94 chaos monkey - every 1 in 5 fails > - 0.96 - 1 in 10 fails. > > machines don't come backup. > > 4 tests are good > - test too many regions > - want tests to scale to scale > - want to kill less -- not completely random. > > bigtop does this > - put in maven. > > ----- > > recover lease thing > > ---- > > locality improvements > > ---- > > Hadoop summit > - day before - birds of a feather > > ---- > > compactions > > -- > // Jonathan Hsieh (shay) > // Software Engineer, Cloudera > // [email protected] >
