Eric Payne updated YARN-3978:
    Attachment: YARN-3978.004.patch

Thanks very much [~jeagles] for your review and comments.

It is not evident to me the changes made to TestClientRMService, 
TestChildQueueOrder, TestLeafQueue, TestReservations, and TestFifoScheduler. 
Could you help to explain those changes or remove them if they are extra?
In each of these tests, the {{rmcontext}} is mocked before constructing a new 
instance of {{RMContainerImpl}}, but the {{getYarnConfiguration}} method is not 
handled. Since this patch adds a dependency on 
{{rmContext.getYarnConfiguration()}} in the constructor for 
{{RMContainerImpl}}, an explicit mock for {{getYarnConfiguration}} had to be 
added in these tests to prevent NPE.
Please update the comment
+ // Store system metrics for all containers only when storeContainerMetaInfo
+ // is true.
To indicate that AM metrics publishing are delayed until later in this scenario.
Is there a better configuration name that could be used? 
save-container-meta-info doesn't convey that AM container info is still 
published if this flag is disabled.
How about {{save-non-am-container-meta-info}}? I thought about 
{{save-only-am-container-meta-info}}, but then {{true}} would mean that 
publishing of non-am containers would be turned off, and I thought that was too 

> Configurably turn off the saving of container info in Generic AHS
> -----------------------------------------------------------------
>                 Key: YARN-3978
>                 URL: https://issues.apache.org/jira/browse/YARN-3978
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: timelineserver, yarn
>    Affects Versions: 2.8.0, 2.7.1
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>         Attachments: YARN-3978.001.patch, YARN-3978.002.patch, 
> YARN-3978.003.patch, YARN-3978.004.patch
> Depending on how each application's metadata is stored, one week's worth of 
> data stored in the Generic Application History Server's database can grow to 
> be almost a terabyte of local disk space. In order to alleviate this, I 
> suggest that there is a need for a configuration option to turn off saving of 
> non-AM container metadata in the GAHS data store.

This message was sent by Atlassian JIRA

Reply via email to