Agreed Luke. Thanks for pointing it out, I'll track it as such. Arun
On Apr 26, 2013, at 1:37 PM, Luke Lu wrote: > If protocol compatibility of v2 and v3 is a goal, HADOOP-8990 should be a > blocker for v2. > > __Luke > > On Fri, Apr 26, 2013 at 12:07 PM, Eli Collins <e...@cloudera.com> wrote: > >> On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy <a...@hortonworks.com> >> wrote: >>> >>> On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote: >>> >>>> On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy <a...@hortonworks.com> >> wrote: >>>> >>>>> With that in mind, I really want to make a serious push to lock down >> APIs and wire-protocols for hadoop-2.0.5-beta. >>>>> Thus, we can confidently support hadoop-2.x in a compatible manner in >> the future. So, it's fine to add new features, >>>>> but please ensure that all APIs are frozen for hadoop-2.0.5-beta >>>> >>>> Arun, since it sounds like you have a pretty definite idea >>>> in mind for what you want 'beta' label to actually mean, >>>> could you, please, share the exact criteria? >>> >>> Sorry, I'm not sure if this is exactly what you are looking for but, as >> I mentioned above, the primary aim would be make the final set of required >> API/write-protocol changes so that we can call it a 'beta' i.e. once >> 2.0.5-beta ships users & downstream projects can be confident about forward >> compatibility in hadoop-2.x line. Obviously, we might discover a blocker >> bug post 2.0.5 which *might* necessitate an unfortunate change - but that >> should be an outstanding exception. >> >> Arun, Suresh, >> >> Mind reviewing the following page Karthik put together on >> compatibility? http://wiki.apache.org/hadoop/Compatibility >> >> I think we should do something similar to what Sanjay proposed in >> HADOOP-5071 for Hadoop v2. If we get on the same page on >> compatibility terms/APIs then we can quickly draft the policy, at >> least for the things we've already got consensus on. I think our new >> developers, users, downstream projects, and partners would really >> appreciate us making this clear. If people like the content we can >> move it to the Hadoop website and maintain it in svn like the bylaws. >> >> The reason I think we need to do so is because there's been confusion >> about what types of compatibility we promise and some open questions >> which I'm not sure everyone is clear on. Examples: >> - Are we going to preserve Hadoop v3 clients against v2 servers now >> that we have protobuf support? (I think so..) >> - Can we break rolling upgrade of daemons in updates post GA? (I don't >> think so..) >> - Do we disallow HDFS metadata changes that require an HDFS upgrade in >> an update? (I think so..) >> - Can we remove methods from v2 and v2 updates that were deprecated in >> v0.20-22? (Unclear) >> - Will we preserve binary compatibility for MR2 going forward? (I think >> so..) >> - Does the ability to support multiple versions of MR simultaneously >> via MR2 change the MR API compatibility story? (I don't think so..) >> - Are the RM protocols sufficiently stable to disallow incompatible >> changes potentially required by non-MR projects? (Unclear, most large >> Yarn deployments I'm aware of are running 0.23, not v2 alphas) >> >> I'm also not sure there's currently consensus on what an incompatible >> change is. For example, I think HADOOP-9151 is incompatible because it >> broke client/server wire compatibility with previous releases and any >> change that breaks wire compatibility is incompatible. Suresh felt it >> was not an incompatible change because it did not affect API >> compatibility (ie PB is not considered part of the API) and the change >> occurred while v2 is in alpha. Not sure we need to go through the >> whole exercise of what's allowed in an alpha and beta (water under the >> bridge, hopefully), but I do think we should clearly define an >> incompatible change. It's fine that v2 has been a bit wild wild west >> in the alpha development stage but I think we need to get a little >> more rigorous. >> >> Thanks, >> Eli >> -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
