Maysam Yabandeh commented on YARN-1530:

bq. YARN apps already depend on ZK/RM/HDFS being up. Every new service 
dependency we add will only increase the chances of YARN apps failing or 
slowing down. That's true even if the ATS service's uptime is as good as ZK or 
bq. Realistically, getting the ATS service's uptime to the same level as ZK or 
HDFS is a long and winding road. Especially when most discussions here assume 
HBase as the backing store. HBase's uptime is lower than HDFS/ZK/RM because 
it's more complex to operate. If HBase going down means ATS service going down, 
then we certainly should guard against this failure scenario.


bq. And if we have a choice to decouple the write path from the ATS service, 
why not?
bq. If we have an alternate code path to persist events first before they hit 
the final backing store, why not do that all the time?

I would call that a reasonable approach. One alternative also is to use HDFS as 
the backup plan, i.e., use it when HBase is down. Anyway, having ATS pluggable 
I guess all approaches can grow independently.

> [Umbrella] Store, manage and serve per-framework application-timeline data
> --------------------------------------------------------------------------
>                 Key: YARN-1530
>                 URL: https://issues.apache.org/jira/browse/YARN-1530
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>         Attachments: ATS-Write-Pipeline-Design-Proposal.pdf, 
> ATS-meet-up-8-28-2014-notes.pdf, application timeline design-20140108.pdf, 
> application timeline design-20140116.pdf, application timeline 
> design-20140130.pdf, application timeline design-20140210.pdf
> This is a sibling JIRA for YARN-321.
> Today, each application/framework has to do store, and serve per-framework 
> data all by itself as YARN doesn't have a common solution. This JIRA attempts 
> to solve the storage, management and serving of per-framework data from 
> various applications, both running and finished. The aim is to change YARN to 
> collect and store data in a generic manner with plugin points for frameworks 
> to do their own thing w.r.t interpretation and serving.

This message was sent by Atlassian JIRA

Reply via email to