[ 
https://issues.apache.org/jira/browse/YARN-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14166032#comment-14166032
 ] 

Carlo Curino commented on YARN-2672:
------------------------------------

The patch contains:

 *  a simple synthetic generator, that allows to control avg and stdev for many 
common parameters (#maps, #reducers, map-time, red-time, IOs for 
in/out/shuffle, deadlines and duration of jobs for reservation).  It is easy to 
define (see syn.json) different workloads, and job classes with various 
properties (size/frequency). 
Not very tested is also the possibility to generate jobs at different rates (by 
controlling the "weighting" of subsequent time ranges).  This is generally 
useful.

 * Extensions to experiment with reservations, where we can specify with what 
probability a job class would be run "with reservation", and submit the 
corresponding ReservationRequeust via YARN-1051, and upon acceptance launch the 
job in it. 

The patch is *rough*, but as few people are starting to experiment with 
YARN-1051 we thought it was important to toss the code out and let folks 
experiments/improve/provide feedback. 

> Improve Gridmix (synthetic generator + reservation support)
> -----------------------------------------------------------
>
>                 Key: YARN-2672
>                 URL: https://issues.apache.org/jira/browse/YARN-2672
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, fairscheduler, resourcemanager
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>         Attachments: YARN-2672.patch
>
>
> This JIRA proposes an enhancement of Gridmix that contains:
> 1) a synthetic generator to produce load without the need of a trace, but 
> based on distributions
> 2) include negotiation of reservations (to test YARN-1051). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to