This is great news. A few things to consider before doing the merge: 1* The merge must be done in trunk first (then, ideally, from trunk into branch-2) 2* The last update of YARN-321 was done in NOV10, this was done from branch-2 (that seems a NIT as it should be against trunk) 3* IMO, until we don’t have security, we should merge into trunk only
I would like to see #1 and #2 taken care before making a decision. The reason for this is that if the source changes outside of the AHS are too pervasive, then we may end up be making difficult backports from trunk to branch-2 and release branches because of the delta. Can we get a rebase of the YARN-321 to the head of trunk to see if my concerns are valid or not? Thanks. Alejandro On Mon, Jan 6, 2014 at 1:11 AM, Sandy Ryza <sandy.r...@cloudera.com> wrote: > Very excited for this feature and appreciative of all the work put into > this. Reviewed the JIRA and my only two remaining concerns are about > documentation and API stability. > > Regarding doc, while we don't necessarily need full documentation before > merging, my feeling is that we should at least have a page on it that will > allow give cluster operators a sketch of its role, APIs, and implications. > If this doesn't exist yet, would be happy to help with reviewing. > > Regarding APIs, I imagine some will jump on this as soon as it becomes part > of a release as it's a fairly essential feature for non-MR apps. My > opinion is that we should mark each API @Stable unless there is a strong > reason for it not to be. When I say APIs, I am thinking of the REST APIs, > RPC interface, and RM<->AHS shared-bus. > > -Sandy > > > On Sun, Jan 5, 2014 at 8:33 PM, lohit <lohit.vijayar...@gmail.com> wrote: > > > +1 to merge into branch2 now. > > On Jan 5, 2014 6:22 PM, "Zhijie Shen" <zs...@hortonworks.com> wrote: > > > > > Hi folks, > > > > > > Majority of the functionality of Application History Server has been > > > completed on branch YARN-321. AHS can now work end-to-end. > > ResourceManager > > > records the historical information of the application, the application > > > attempt and the container in terms of events via a history writer on a > > > separate thread. The historical information is going to be persisted on > > > HDFS. On the other side, an application history server runs as a > separate > > > process, and it allows users to access the historical information via > RPC > > > interface, web UI and REST APIs. > > > > > > According to the proposal, the only major missing piece is security. > > Except > > > it, YARN-321 should be pretty much ready to be merged to Branch-2. We > > > propose to merge YARN-321 to Branch-2 now, such that we can prevent > > > YARN-321 from falling behind too much, and reduce the effort of editing > > > merge conflicts. After merge, we can continue on security, refactoring > > > duplicate code, fixing bugs and other improvements. > > > > > > Anyone who is interested in looking at AHS can review/play with > YARN-321 > > > branch: http://svn.apache.org/viewvc/hadoop/common/branches/YARN-321/ > > > > > > You can also have a look at the latest design doc: > > > > > > > > > https://issues.apache.org/jira/secure/attachment/12619638/Generic%20Application%20History%20-%20Design-20131219.pdf > > > > > > If there are no objections, we'll push towards updating the branch, > > running > > > the patch through Jenkins and getting ready for merge vote by the end > of > > > this week. > > > > > > Thanks, > > > Zhijie > > > > > > -- > > > Zhijie Shen > > > Hortonworks Inc. > > > http://hortonworks.com/ > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > > -- Alejandro
