On May 15, 2013, at 2:54 PM, Roman Shaposhnik wrote: > This is not my argument at all. I apologize if somehow I failed to > communicate it, but here's what my argument boils down to: > given *my* experience with Hadoop 2.0.x series and Bigtop > release every time I try a different release of Hadoop 2.0.x > I run into issues that scare me. They scare me because > they are so basic yet they make component like Sqoop > and Oozie (and I believe Giraph on one occasion) > pretty much DOA for YARN-base mapreduce implementation.
Why they should *scare* *you*? As Stevel and Arun also pointed out on others mails in this thread, we have no way of finding all the bugs. What is basic in your environment isn't basic in mine. Just today I ran into a *basic* issue of not being able to run secure oozie setup on top of one of the stable 1.x release in one of the usual environments. Now if I share your level of concern, I should be *scared* about why none of the testing we did in the past 3 years manage to uncover it. Or may be why BigTop is not able to help find these issues for us if at all. > In my mind, what that translates into is the fact that nobody > did *any* real testing of a particular downstream component > running on a given Hadoop 2.0.x release. Like I said -- > the issues so far make the components in question DOA. Thanks for *finally* adding this bit "of a particular downstream component". It is very likely none of us tested Sqoop on top of 2.0.3/4-alpha. But you know what, if BigTop didn't exist, someone from Sqoop community would have nudged us. And nobody will really need to be *scared* - all of this is expected in the alpha life cycle. > How many more issues like that one (regardless of how > they originated) are in branch-2? Wouldn't we want to > know before declaring Hadoop 2.0.5 beta? We want to know, and you know what, we already know some of them. Lots of them. You can find them in issue tracker if you ever want to. For example in YARN: https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+YARN+AND+status+%3D+Open+ORDER+BY+priority+DESC&mode=hide There are a total of 258 open tickets, 10 critical, 253 major (and if categorized will turn into critical and blocker issues). And particularly 23 against 2.0.4-alpha and 34 against 2.0.3-alpha. So, please. With due respect, BigTop isn't the only project discovering bugs which make downstream components DOA. Thanks, +Vinod
