Re: Differences between HistoryServer and Yarn TimeLine server?

2014-04-25 Thread Ashwin Shankar
Thanks Zhijie ! I had few more questions : 1. I played around with the timeline server ui today which showed the generic application history details, but I couldn't find any page for application specific data. Is the expectation that every application needs to build their own UI using the exposed

Re: Differences between HistoryServer and Yarn TimeLine server?

2014-04-25 Thread Zhijie Shen
1 and 4: We have thought about that in addition to service application specific data, the timeline server should accept the web UI plugin from the application, install it and render the data on the web page according the application's design, but still need to figure out the plan. Before that, the

Re: Differences between HistoryServer and Yarn TimeLine server?

2014-04-24 Thread Ashwin Shankar
Hi Zhijie, There seems to two umbrella jiras for this - YARN-321 and YARN-1530,can you please let me know what is the difference ? Is timeline server finally going to be YARN321+YARN1530 ? You mentioned that MR is going to integrated with timeline server,is there a jira I can watch ? Thanks,

Re: Differences between HistoryServer and Yarn TimeLine server?

2014-04-24 Thread Zhijie Shen
Ashwin, YARN-321 focuses on the issue in the scope of generic application history service, while YARN-1530 covers the framework specific data service. And yes, the timeline server is going to cover both. We've not such a Jira before, but it is described in YARN-321's design doc. Anyway, I open a

Re: Differences between HistoryServer and Yarn TimeLine server?

2014-04-23 Thread sam liu
Zhijie, I am much clear now. Thanks a lot! As my understanding, besides previous Job History Server, hadoop now has a new timeline server which could restore both the generic YARN application history and the framework specific information. However, I think the timeline server also include the

Re: Differences between HistoryServer and Yarn TimeLine server?

2014-04-23 Thread Zhijie Shen
Sam, You're right. We can definitely integrate MapReduce to use the timeline server to store and serve its specific data, and this is actually our plan. However, it's a big move, and we still need time to get it done. In addition, not to disturb the users that are currently relying on JHS for MR

Differences between HistoryServer and Yarn TimeLine server?

2014-04-22 Thread sam liu
Hi Experts, I am confusing on these two concepts. Could you help explain the differences? Thanks!

Re: Differences between HistoryServer and Yarn TimeLine server?

2014-04-22 Thread Zhijie Shen
In Hadoop 2.4, we have delivered the timeline server at a preview stage, which actually can serve some generic YARN application history as well as the framework specific information. Due to the development logistics, we have created the two concepts: History Server and Timeline Server. To be