+1 for this idea. It will provide better isolation for scheduler simulating to split NM simulators or/and AM simulators to different daemons, which can be placed to different hosts.
On Wed, Dec 20, 2017 at 2:18 AM Jasson Chenwei <ynjassionc...@gmail.com> wrote: > It is an interesting point. So you mean you will set up like two daemons, > one for RM(acts for resource allocation) and the other for NMs(each thread > represents for a NM ?). I am also working on a large scale cluster > simulation project now. I am interested in your configuration for your > simulation(machine capacity, machine load) and your bottleneck (like if the > performance is bottlenecked by CPU or the lock-contention or the other > resources)??? > > > Wei Chen > > On Mon, Dec 18, 2017 at 11:54 PM, zhangshilong (JIRA) <j...@apache.org> > wrote: > > > zhangshilong created YARN-7672: > > ---------------------------------- > > > > Summary: hadoop-sls can not simulate huge scale of YARN > > Key: YARN-7672 > > URL: https://issues.apache.org/jira/browse/YARN-7672 > > Project: Hadoop YARN > > Issue Type: Improvement > > Reporter: zhangshilong > > Assignee: zhangshilong > > > > > > Our YARN cluster scale to nearly 10 thousands nodes. > > We need to do scheduler pressure test. > > we start 2000+ threads to simulate NM and AM. So cpu.load very high to > > 100+. I thought that will affect performance evaluation of scheduler. > > So I thought to separate the scheduler from the simulator. > > I start a real RM. Then SLS will register nodes to RM,And submit apps to > > RM using RM RPC. > > > > > > > > -- > > This message was sent by Atlassian JIRA > > (v6.4.14#64029) > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > -- Best, Yufei
