I think we should not require Job Q compatibility for 1.0 release. thanks, dhruba
On Mon, Sep 28, 2009 at 11:06 AM, Sanjay Radia <sra...@yahoo-inc.com> wrote: > > On Sep 28, 2009, at 3:15 AM, Steve Loughran wrote: > > Dhruba Borthakur wrote: >> > It is really nice to have wire-compatibility between clients and servers >> > running different versions of hadoop. The reason we would like this is >> > because we can allow the same client (Hive, etc) submit jobs to two >> > different clusters running different versions of hadoop. But I am not >> stuck >> > up on the name of the release that supports wire-compatibility, it can >> be >> > either 1.0 or something later than that. >> > API compatibility +1 >> > Data compatibility +1 >> > Job Q compatibility -1Wire compatibility +0 >> >> >> That's stability of the job submission network protocol you are looking >> for there. >> * We need a job submission API that is designed to work over long-haul >> links and versions >> * It does not have to be the same as anything used in-cluster >> * It does not actually need to run in the JobTracker. An independent >> service bridging the stable long-haul API to an unstable datacentre >> protocol does work, though authentication and user-rights are a >> troublespot >> >> > > > I think you are misinterpreting what Job Q compatibility means. > It is about jobs already in the queue surviving an upgrade across a > release. > > See my initial proposal on Jan 16th: > > https://issues.apache.org/jira/browse/HADOOP-5071?focusedCommentId=12664691&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel > #action_12664691 > > Doug argued that it is nice to have but not required for 1.0 - can be added > later. > > > sanjay > > >> Similarly, it would be good for a stable long-haul HDFS protocol, such >> as FTP or webdav. Again, no need to build into the namenode . >> >> see http://www.slideshare.net/steve_l/long-haul-hadoop >> and commentary under http://wiki.apache.org/hadoop/BristolHadoopWorkshop >> >> > -- Connect to me at http://www.facebook.com/dhruba
