Naganarasimha G R commented on YARN-3034:

Hi [~gtCarrera9], thanks for reviewing the patch,
# _point1_
I think there is difference in understanding about the approach here. Based on 
discussions with [~sjlee0] and in the design doc, 
_In section 4.1_
RM itself has its own ATS process to be able to write RM-specific timeline 
events (e.g.
application lifecycle events). RM can also use YARN tags to associate events 
with a specific
flow/run/app. The volume of data coming directly from RM should not be great.
IIUC RM has its own single ATS aggregator(service) and writer and it differs 
from NM, where NM through service discovery <YARN-3039> identifies 
AppLevelAggregatorService and posts the entities through it.
# _point2_
Yes, i agree to your point here i could have kept these modifications separate 
from this jira, i got similar kind of comment from Sangjin where in he was 
asking both old and new ATS should be working and based on configuration we 
pick the appropriate ATS notifications from RM. Will take care in next patch.
# _point3_
Well i tried to keep it in sync with the existing ATS code 
(SystemMetricsPublisher). Once my queries are clarified then thought about 
discussing the package structure in yarn-3166.
Currently i have following queries :
# Will RM have its own Aggregator (which i feel is correct as we are publishing 
only app and app attempt life cycle events from RM,) or collection of 
application level aggregators ( it doesn't serve any purpose for having this 
separately in RM). As per YARN-3030, Using AUX services separate 
AppLevelAggregatorService is created per App in each NM.
# If your understanding is correct (collection of application level aggregators 
for both RM and NM). Then i have few queries based on YARN-3030.
#* Why are we starting AppLevelAggregatorService in NM through Aux services,  
we should have created this from RM. so that initial app life cycle events can 
be posted to ATS.
#* whats the scope of RMTimelineAggregator when we have 
# If My understanding is correct (RM have its own ATS Aggregator ) i have 
following queries :
#* Based on the discussions we had on Feb11,  I understand that RM and NM 
should not be directly dependent on TimelineService. But in 3030 patch, 
BaseAggregatorService.java is in timeline service project hence where to place 
this RMTimelineAggregator.java class (as it extends BaseAggregatorService ) ? 
#* If we plan to handle similar to current approach i.e send the Entity data 
through a rest client to a timeline writer service(RMTimelineAggregator), where 
should this service be running i.e. as part of which process or should it be a 
daemon on its own?

> [Aggregator wireup] Implement RM starting its ATS writer
> --------------------------------------------------------
>                 Key: YARN-3034
>                 URL: https://issues.apache.org/jira/browse/YARN-3034
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3034.20150205-1.patch
> Per design in YARN-2928, implement resource managers starting their own ATS 
> writers.

This message was sent by Atlassian JIRA

Reply via email to