Hi Zhijie, There seems to two umbrella jiras for this - YARN-321 and YARN-1530,can you please let me know what is the difference ? Is timeline server finally going to be YARN321+YARN1530 ?
You mentioned that MR is going to integrated with timeline server,is there a jira I can watch ? Thanks, Ashwin On Wed, Apr 23, 2014 at 10:15 PM, Zhijie Shen <[email protected]> wrote: > Sam, > > You're right. We can definitely integrate MapReduce to use the timeline > server to store and serve its specific data, and this is actually our plan. > > However, it's a big move, and we still need time to get it done. In > addition, not to disturb the users that are currently relying on JHS for MR > job information, we cannot simply remove JHS from Hadoop. > > > On Wed, Apr 23, 2014 at 8:15 PM, sam liu <[email protected]> wrote: > >> Zhijie, >> >> I am much clear now. Thanks a lot! >> >> As my understanding, besides previous Job History Server, hadoop now has >> a new timeline server which could restore both the generic YARN application >> history and the framework specific information. However, I think the >> timeline server also include the functions of Job History Server, because >> it can store the framework specific information(of course, include >> mapreduce framework). In another words, Job History Server is not necessary >> any more.* If that's the case, why hadoop still include Job History >> Server?* >> >> >> 2014-04-23 12:56 GMT+08:00 Zhijie Shen <[email protected]>: >> >>> In Hadoop 2.4, we have delivered the timeline server at a preview >>> stage, which actually can serve some generic YARN application history as >>> well as the framework specific information. Due to the development >>> logistics, we have created the two concepts: History Server and Timeline >>> Server. To be simple, you can consider the history server of the service of >>> the generic YARN application information, while consider the timeline >>> server of the service of the framework specific information. Importantly, >>> we just have one daemon, which includes both services, and which we'd like >>> to call timeline server (unfortunately, the confusing thing is that the >>> command to start the daemon is "historyserver"). We're going on working on >>> the timeline server to integrate these two parts, including refactoring the >>> names. >>> >>> BTW, if you mean MapReduce JobHistoryServer by HistoryServer, it's a >>> different daemon, which serves the historic information of MapReduce jobs >>> only. >>> >>> >>> On Tue, Apr 22, 2014 at 8:44 PM, sam liu <[email protected]> wrote: >>> >>>> Hi Experts, >>>> >>>> I am confusing on these two concepts. Could you help explain the >>>> differences? >>>> >>>> Thanks! >>>> >>> >>> >>> >>> -- >>> Zhijie Shen >>> Hortonworks Inc. >>> http://hortonworks.com/ >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >> >> >> > > > -- > Zhijie Shen > Hortonworks Inc. > http://hortonworks.com/ > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > -- Thanks, Ashwin
