Hi a) The two big commits to the master (YARN integration and feeds) happened as atomic units that makes it easier to reset the master to the version prior to each feature and verify if the build began showing OOM after each of the suspected commits. We have a pretty deterministic way of nailing down the commit that introduced the problem. I would suggest, instead of disabling the feeds tests, can we revert to the earlier commit and confirm if the feeds commit did introduce the behavior and repeat the test with the YARN commit that followed. We should be able to see sudden increase/drop in build stability by running sufficient number of iterations.
b) I have not been able to reproduce the OOM at my setup where I have been running the build repeatedly. @Ian are you able to reproduce it at your system? May be I am not running the build sufficient number of times? I am still not able to understand how removal of test cases still causes the OOM? I can go back and look at the precise changes made during the feeds commit that could introduce OOM even if feeds are not involved at all, but as I see it, the changes made do not play a role if feeds are not being ingested. Regards, Raman On Thu, Jul 2, 2015 at 6:42 PM, Ian Maxon <[email protected]> wrote: > Hi all, > > We are close to having a release ready, but there's a few things left > on the checklist before we can cut the first Apache release. I think > most things on this list are underway, but I'll put them here just for > reference/visibility. Comments and thoughts are welcomed. > > - Build stability after merging YARN and Feeds seems to have seriously > declined. It's hard to get a build to go through to the end without > going OOM at all now honestly, so this is a Problem. I think it may be > related to Feeds, but even after disabling the tests > (https://asterix-gerrit.ics.uci.edu/#/c/312/), I still see it. > Therefore I am not precisely sure what is going on, but it only > started to happen after we merged those two features. It's not exactly > obvious to me where the memory leak is coming from. @Raman, it would > be great to get your advice/thoughts on this. > > - Metadata name changes and Metadata caching consistency fixes are > underway by Ildar. > > - The repackaging and license checker patches still need to be merged > in, but this should happen after the above two features are merged. > They are otherwise ready for review though. > > - Now that Feeds is merged, the Apache website should be changed to > the new version that has been in draft form for a few weeks now. > Before it may have been a little premature, but now it should be > accurate. The documentation site should also be reverted to its prior > state, before it was quickly patched to serve as an interim website. > > > If there's anything else I am missing that should be in this list, > please feel free to add it into this thread. > > Thanks, > -Ian > -- Raman
