Thanks for the summary Deveraj. The webex recording for the meeting is at:
https://cloudera.webex.com/cloudera/ldr.php?AT=pb&SP=MC&rID=120418137&rKey=d058765b798c47c5 Please let me know if you cannot access it. Regards, - Dave On Tue, Feb 19, 2013 at 5:30 PM, Jean-Marc Spaggiari < [email protected]> wrote: > Awesome! > > Thanks a for those notes Devaraj! > > Very useful for the unfortunates who did not got the chance to join the > meeting ... > Le 19 févr. 2013 20:27, "Devaraj Das" <[email protected]> a écrit : > > > - Stabilizing 0.96 / CI > > -- not running continuously > > -- Roman: is it a good idea to run IT under Bigtop > > - the HBase unit tests are good.. > > - What about HBase as a backend for hive - tests for those > > - Do we think there is value with the integration with the rest > > of the hadoop ecosystem > > -- Jon: What's needed to make this work > > -- Roman: Collective agreement that we want to solve this > > -- Stack: We need to run the IT tests from Enis and Sergey running > > continuously > > -- Roman: If we all agree that this is something needs to be fixed > > .. then yes we would talk about mechanics. Bigtop doesn't have > > expertise in HBase and hence HBase folks would have to debug failures. > > -- Roman: Bigtop is committed to tests. Less than a dozen tests for > > hbase currently.. > > -- We have been running validation around RC time. Find all kinds of > > issues - sometimes trivial (maybe a config issue), > > -- Roman can offer CI for trunk but will work for only hadoop-2 line. > > -- Roman does first line of triage. > > -- No issues to do with other ecosystem artifacts. Bigtop ensures > > the right artifacts are in place. > > -- hadoop-2 is important but not particular about the version of > > hadoop within 2. For example 2.0.2 > > -- Gary: Will be good to run security tests > > -- Roman & Devaraj to talk on how this can be done/implemented > > > > On 0.96 branching > > - Lars: > > -- We will have three branches to maintain. > > - Stack: we need to stabilize quickly > > - Enis: What about 0.95. > > - Stack: Just do the snapshots thing. Every week, give a snapshot > > - Enis: we've done a bunch of stuff in RPC.. If we have to break > > something, we can if it is beta (0.96-beta). > > - Agreement is there generally ... Debate on the name with snapshot > > versus 95.0/96.0 > > -- 0.95 experimental .. 0.96 will be stable > > -- If we go with 0.95, releasing will be easier as well.. > > -- 0.95 will not be in production.. > > -- 0.96 will be off 0.95 branch and not trunk based > > > > - How do we go about committing issues.. (issue commit rate is low) > > -- 0.94 is stable - 2 new commits and 2 new bugs a day > > -- 0.96 has lots of issues not reviewed > > -- Break up the patch into multiple smaller pieces to make review easier > > -- Branching on a big feature was suggested - > > --- Issues: committer needs to be there > > Sergey: If the rate of change is high on common code (to the > > branches) then merging will be tough > > --- Jon: Refactor should be done in the main branch (since it > > doesn't add any new funtionality) > > --- Release often to reduce #backports overall and issues with that.. > > > > - Review process .. how to drive a review to closure. Effort goes > > waste if the review process is not completed. The same reviewer should > > continue to review the patch .. > > - Hard to enforce any process > > - Enis: there should be a summary of the patch and all that .. so that > > the review process is helped.. Hard to understand the architecture of > > the patch unless documented > > - Jon: It should be easy to make a one-to-one correspondence between > > the description and the patch > > - The commit should have only the jira# as opposed to pages of > description > > > > - Component owners: is this working. Committers need to be forthcoming > > with reviews > > -- Maybe review the modules and add some more if needed. > > > > -- Good that we have more contributions coming than we have > > reviewers, but unless we keep up, we will plateau > > -- Mail on dev@ list if review doesn't happen > > > > - Dev co-ordination: > > -- How best can we pull together > > -- Priorities: > > --- Getting 0.96 out is priority > > --- Backports to 0.94 will happen .. until 0.96 is stable > > --- 0.92 release ? Any committer who wants to make a release can do > > so (maybe with some backports, etc.) > > --- Backporting can be tough if there are bugs and the bugfix has > > to be applied to all branches > > > > - HBASE-2600 - this requires a change in the client and the server. > > They have to be changed in lock-step. Its hard to do this .. Jon > > doesn't want to have the fix for 0.96. So 0.98 might be another > > singular release. Maybe do a rewrite of the meta after taking a lock > > on the meta, do a shim layer to handle the backward compat. between > > 0.96 and 0.98 > > > > - What do people want to get into 0.94 > > -- The biggest thing - Snapshots, mostly new code, about a 3rd of the > > stuff in 0.94 already > > -- Compactions improvments - no backport > > > > - How devs can better co-ordinate > > -- Snapshots co-ordination working well > > -- One page design is useful (makes it readable and all) > > -- How about handling the stripe compaction - where an idea leads to > > a bunch of others > > -- Again write-up should be done > > > > - Should we change the description to match the comments > > -- Two ideas suggested: > > --- We probably should have the description updated with the > > "Date: new description" if the issue at hand is updated > > --- Should we have a summary after a bunch of comments - yes > > > > - The face-to-face meetings are useful. We should semd out the minutes > > of the discussion to the dev list. We probably should have more > > focused huddles. Discuss but don't decide (decide on the jira)! > > > > - Jon: Would people be amenable to merge sooner rather than later on > > snapshots? Tested and being beaten up. > > - Stack: Yes > > > > - What else goes in in 0.96: > > -- RPC refactor > > -- ROOT removal > > -- Compaction stuff > > -- Package name mailing list thread - there is now a jira on that. > > We shouldn't break clients. Package name changes is not worth the > > trouble. > > > > - A bunch of discussions on the RPC with KeyValue/Cells > > > > - What do we do about usability > > -- It'll be nice if we don't need to change configs.. > > -- Maybe expose more metrics and then allow for online config > > changes since automatic config is difficult and needs to be battle > > tested and all. > > > > - Benchmarking of the release: > > -- We should measure the overhead of PB stuff > > >
