It is an interesting point. So you mean you will set up like two daemons, one for RM(acts for resource allocation) and the other for NMs(each thread represents for a NM ?). I am also working on a large scale cluster simulation project now. I am interested in your configuration for your simulation(machine capacity, machine load) and your bottleneck (like if the performance is bottlenecked by CPU or the lock-contention or the other resources)???
Wei Chen On Mon, Dec 18, 2017 at 11:54 PM, zhangshilong (JIRA) <[email protected]> wrote: > zhangshilong created YARN-7672: > ---------------------------------- > > Summary: hadoop-sls can not simulate huge scale of YARN > Key: YARN-7672 > URL: https://issues.apache.org/jira/browse/YARN-7672 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: zhangshilong > Assignee: zhangshilong > > > Our YARN cluster scale to nearly 10 thousands nodes. > We need to do scheduler pressure test. > we start 2000+ threads to simulate NM and AM. So cpu.load very high to > 100+. I thought that will affect performance evaluation of scheduler. > So I thought to separate the scheduler from the simulator. > I start a real RM. Then SLS will register nodes to RM,And submit apps to > RM using RM RPC. > > > > -- > This message was sent by Atlassian JIRA > (v6.4.14#64029) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
