Bill,
I'm at a computer now so I'll take a little more time to explain. The
profiles are used by the REST API so that you can tell a the resource
manager to add a new node manager with cpu and memory defined by the
profile. If you set a profile large as:
large:
cpu: 16
mem: 32768
then you could flexup using {profile: large, instances: 10} to get 10
nodemanagers running with 32 GB memory and 16 cpu core capacity. You could
flexdown {instance: 10} to remove the capacity. How to flexup/flexdown is
documented in API.md. You can also flexup/flexdown using the webui that
runs in port 8192.
In the near future we plan to have fine grain scaling, then when you launch
a resource manager you'll also launch instance of node manager with 0 cpu
and 0 mem capacity. As you run yarn applications the node managers will
add cpu and mem capacity to meet your jobs container needs.
Let me know if you have more questions.
Darin
On Thu, Jun 11, 2015 at 6:43 PM, Bill Sparks <[email protected]> wrote:
> I've changed them with no ill effect. Just wondering what the purpose
> is. My yarn configurations will allow much more that the profiles set by
> mesos/myriad. So is myriad scheduler going to bounce the request based on
> profiles or be passive and let yarn containers do the work. All this is
> early days for me.
>
> --
> Jonathan (Bill) Sparks
> Software Architecture
> Cray Inc.
>
> From: Darin Johnson <[email protected]>
> Date: Thursday, June 11, 2015 5:36 PM
> To: "[email protected]" <[email protected]>
> Cc: Bill Sparks <[email protected]>
> Subject: Re: Myriad/YARN configuration
>
> I've changed the profiles before with no ill effect, you should be fine.
>
> On Thu, Jun 11, 2015 at 6:28 PM, Adam Bordelon <[email protected]> wrote:
>
>> +dev@myriad, -user@mesos
>>
>> On Thu, Jun 11, 2015 at 8:51 AM, Bill Sparks <[email protected]> wrote:
>>
>> > I have a question about the profiles configured
>> > in myriad-config-default.yml, do they have to match parameters
>> specified in
>> > yarn-site.xml.
>> >
>> > E.g.
>> > myriad-config-default.yml:
>> > large:
>> > cpu: 4
>> > mem: 4096
>> >
>> > yarn-site.xml:
>> > <property>
>> > <name>yarn.nodemanager.resource.cpu-vcores</name>
>> > <value>${nodemanager.resource.cpu-vcores}</value>
>> > </property>
>> >
>> > <property>
>> > <name>yarn.nodemanager.resource.memory-mb</name>
>> > <value>${nodemanager.resource.memory-mb}</value>
>> > </property>
>> >
>> > I ask because we have large memory/high core count nodes 48 cores/64GB
>> > memory.
>> >
>> > --
>> > Jonathan (Bill) Sparks
>> > Software Architecture
>> > Cray Inc.
>> >
>>
>
>
