Re: [smartos-discuss] New host and CPU question

[Patrick Mooney]

Being a hardware-level hypervisor, the techniques used by Bhyve aren't
relevant to OS-virtualized containers.

>From what I can tell about LXC, it supports mechanisms such as CPU sets and
assigning CPU shares.  While CPU sets _would_ solve the issue of visible
CPUs matching expectations for function inside a zone, it's very
unattractive for achieving good economics in a multi-tenant environment.
The latter mechanism, assigning CPU shares, closely resembles how SmartOS
does CPU resource control today (limiting the amount of CPU time that can
be used, but not the CPUs on which one can spend it).

On 6 December 2016 at 12:30, Len Weincier  wrote:

>
>
> On 6 December 2016 at 17:42, Patrick Mooney 
> wrote:
>
>>
>> On 6 December 2016 at 08:41, Nahum Shalman  wrote
>>
>>> Additional thoughts:
>>> I wonder how bad it would be to allow the LX brand to alter how many
>>> CPUs the Linux system calls see based on a zone property (probably a new
>>> one...)
>>> My first instinct is "not that bad", though my second instinct is "the
>>> consequences of doing so are subtle and it's probably much worse than I
>>> would think".
>>>
>>
>> This topic comes up fairly often.  It's very difficult to tell a
>> convincing "lie" about how many CPUs are available on the system.  If you
>> were to report only 0-7 cores out of 24, for example, software querying the
>> CPU ID may still see higher identifiers if it's scheduled onto the other
>> socket.  It's also possible that total CPU time enforced by the cap would
>> exceed what would be theoretically possible on the virtual CPUs which have
>> been presented.  (If 8.5 seconds of CPU time were consumed on those "8
>> cores" in 1 second, tools like 'top' might be grumpy.)
>>
>> All in all, "subtle consequences" is an accurate description of why the
>> task is difficult accomplish correctly.
>>
>
> I wonder if its worth looking at LXC / bhyve to see if they have any ideas
> on this ?
>
> Subtle consequences indeed !
> *smartos-discuss* | Archives
> 
>  |
> Modify
> 
> Your Subscription 
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Len Weincier]

On 6 December 2016 at 17:42, Patrick Mooney 
wrote:

>
> On 6 December 2016 at 08:41, Nahum Shalman  wrote
>
>> Additional thoughts:
>> I wonder how bad it would be to allow the LX brand to alter how many CPUs
>> the Linux system calls see based on a zone property (probably a new one...)
>> My first instinct is "not that bad", though my second instinct is "the
>> consequences of doing so are subtle and it's probably much worse than I
>> would think".
>>
>
> This topic comes up fairly often.  It's very difficult to tell a
> convincing "lie" about how many CPUs are available on the system.  If you
> were to report only 0-7 cores out of 24, for example, software querying the
> CPU ID may still see higher identifiers if it's scheduled onto the other
> socket.  It's also possible that total CPU time enforced by the cap would
> exceed what would be theoretically possible on the virtual CPUs which have
> been presented.  (If 8.5 seconds of CPU time were consumed on those "8
> cores" in 1 second, tools like 'top' might be grumpy.)
>
> All in all, "subtle consequences" is an accurate description of why the
> task is difficult accomplish correctly.
>

I wonder if its worth looking at LXC / bhyve to see if they have any ideas
on this ?

Subtle consequences indeed !



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Patrick Mooney]

On 6 December 2016 at 08:41, Nahum Shalman  wrote

> Additional thoughts:
> I wonder how bad it would be to allow the LX brand to alter how many CPUs
> the Linux system calls see based on a zone property (probably a new one...)
> My first instinct is "not that bad", though my second instinct is "the
> consequences of doing so are subtle and it's probably much worse than I
> would think".
>

This topic comes up fairly often.  It's very difficult to tell a convincing
"lie" about how many CPUs are available on the system.  If you were to
report only 0-7 cores out of 24, for example, software querying the CPU ID
may still see higher identifiers if it's scheduled onto the other socket.
It's also possible that total CPU time enforced by the cap would exceed
what would be theoretically possible on the virtual CPUs which have been
presented.  (If 8.5 seconds of CPU time were consumed on those "8 cores" in
1 second, tools like 'top' might be grumpy.)

All in all, "subtle consequences" is an accurate description of why the
task is difficult accomplish correctly.



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Paul Sture]

On 6 Dec 2016, at 15:41, Nahum Shalman wrote:



Yes, software in a zone making inferences based on how many cores it 
can

see is problematic.


...


Currently, yes, though depending on the software in question sometimes 
you

can override the degree of parallelism that it will use by setting an
explicit value.


A couple of examples of that:

In SmartOS Live Build, 
https://wiki.smartos.org/display/DOC/Building+SmartOS+on+SmartOS


export MAX_JOBS=8

and in the build for Julia, https://github.com/JuliaLang/julia

To perform a parallel build, use make -j N and supply the maximum 
number of concurrent processes.




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Nahum Shalman]

Responses to two messages plus additionals thought below.

On Fri, Dec 2, 2016 at 11:32 AM, Len Weincier  wrote:
>
> Thats very useful. Inside the joyent public cloud how many cores do you
> see inside an LX zone ? When I spin up an LZ zone I see all the cores on
> the machine, which makes sense. So some customers see 40 Cores ("Intel(R)
> Xeon(R) CPU E5-2670 v2 @ 2.50GHz") and some customers see 128 cores
> ("Intel(R) Xeon(R) CPU E7-4850 v4 @ 2.10GHz") regardless of the cpu_cap
> setting.
>

Inside the Joyent cloud, LX zones see all the cores on the machine in
question. Same as on your hardware.


> The problem is as Ian said above that when cpu_cap is less than the core
> count its tricky for our customers.
>

Yes, software in a zone making inferences based on how many cores it can
see is problematic.


On Fri, Dec 2, 2016 at 5:19 PM, Ian Collins  wrote:
>
> The problem with the current scheme is when you hit a capped zone with
> something like a parallel compile.  The build tool will see all of the
> cores and happily spin up a compile or two on each.  So on a decent machine
> like the one Len describes, there will be 64 CPU intensive jobs running on
> a machine with a cap one CPU...  The result is the load average goes
> through the roof and the the box falls over.  If you do this in a KVM, the
> KVM will curl up and die.
>

Yes


> The only way I know to properly confine a guest is to use a KVM with
> matching CPU cap and count.
>

Currently, yes, though depending on the software in question sometimes you
can override the degree of parallelism that it will use by setting an
explicit value.

Additional thoughts:
I wonder how bad it would be to allow the LX brand to alter how many CPUs
the Linux system calls see based on a zone property (probably a new one...)
My first instinct is "not that bad", though my second instinct is "the
consequences of doing so are subtle and it's probably much worse than I
would think".

I would look into how to make sure those parallel compiles are using an
appropriate level of parallelism. I imagine the set of applications that
will be misbehaving based on core count while theoretically very large will
in practice be relatively small and helping customers work around those
issues might be the easiest solution of them all.

-Nahum



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Ian Collins]

On 12/ 3/16 03:43 AM, Nahum Shalman wrote:
I like to think about CPU as a resource either under contention (more 
processes need CPU time than CPU cores to go around) or not:


CPU caps limit how much you can use when there's no contention. In the 
Joyent Could it ensures people don't get more CPU time than they pay 
for. (example 1 below)


CPU shares ensure fairness when there is contention. If CPU shares are 
doled out in proportion to memory doled out (Like in the Joyent 
Cloud), when everyone is fighting for CPU, the more memory you're 
paying for, the more CPU cycles you'll get. (example 2 below)


If all the zones on a box belong to the same customer (as might be the 
case on your machine), you could set the cpu caps much higher, and 
depend on the shares for fairness.


Examples for clarification:

1. I fire up a g4-highcpu-2G zone in the Joyent cloud. It gets a CPU 
cap of 100. It happens to land on a freshly deployed compute node with 
no other zones on it.
I then run 2 CPU intensive processes. There is no contention so the 
machine is totally capable of scheduling each of them on a separate 
CPU and running them full throttle.
However, because I'm only paying for 1 1vCPU I hit that cap of 100 and 
my two processes have to take turns running. If my workload is really 
that CPU intensive I should scale the zone up to g4-highcpu-4G which 
gets 2vCPUs and a cap of 200.




Hello Nahum,

The problem with the current scheme is when you hit a capped zone with 
something like a parallel compile.  The build tool will see all of the 
cores and happily spin up a compile or two on each.  So on a decent 
machine like the one Len describes, there will be 64 CPU intensive jobs 
running on a machine with a cap one CPU...  The result is the load 
average goes through the roof and the the box falls over.  If you do 
this in a KVM, the KVM will curl up and die.


The only way I know to properly confine a guest is to use a KVM with 
matching CPU cap and count.


Cheers,

--
Ian.



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Len Weincier]

On 2 December 2016 at 18:30, Robert Mustacchi  wrote:

> On 12/2/16 8:17 , Len Weincier wrote:
> > On 2 December 2016 at 18:04, Robert Mustacchi  wrote:
> >
> >> On 12/1/16 23:44 , Len Weincier wrote:
> 
>  This is the classic symptom of a machine with a cpu_cap less than its
> >> CPU
>  (core) count.
> 
> >>>
> >>> So the vm sees all the cores and dispatches threads / processes for
> that
> >>> many cores but the actual uasage is limited by the cpu_cap so the
> threads
> >>> are stalling waiting for the OS to give them a slot to run hence the
> load
> >>> avg climbing iiuc. Setting cpu_cap=0 sets the machine free and its
> >>> responsive
> >>>
> >>> Whats the right way to share the cores for LX brand machines then
> seeing
> >> as
> >>> we cant limit the access to the cores ? In triton is see the default
> >>> packages set the cpu_cap as well which would result in the same issue ?
> >>
> >> At the end of the day you need to make sure you actually tune things
> >> inside of your zones to respect sizing based on what you've set in caps
> >> and shares.
> >>
> >> Robert
> >>
> >
> > Hi Robert
> >
> > Thanks for the reply.   These are machine we are provisioning for
> customers
> > and we unfortunately have very little control over what customers are
> doing
> > inside the zones. The ticket was logged by a customer saying that their
> app
> > had ground to a halt after we moved them to the new hardware.
> >
> > afaics this will happen with triton as well. We will be testing this in a
> > lab to confirm.
>
> It will. There's nothing in Triton that lies about the number of CPUs
> available. Barring something like that, which is going to be very, very
> hard to tell and not confuse things, the solution is tune things in
> applications.
>
> Most applications that tune based on visible CPUs aren't designed to
> scale to the number you have with a 4 socket machine.
>
> Robert
>

We will have to figure out a way to pass this message  on to customers
then. So you have this issue in the joyent public cloud as well ?

Thanks
Len



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Len Weincier]

On 2 December 2016 at 16:43, Nahum Shalman  wrote:

> I like to think about CPU as a resource either under contention (more
> processes need CPU time than CPU cores to go around) or not:
>
> CPU caps limit how much you can use when there's no contention. In the
> Joyent Could it ensures people don't get more CPU time than they pay for.
> (example 1 below)
>
> CPU shares ensure fairness when there is contention. If CPU shares are
> doled out in proportion to memory doled out (Like in the Joyent Cloud),
> when everyone is fighting for CPU, the more memory you're paying for, the
> more CPU cycles you'll get. (example 2 below)
>
> If all the zones on a box belong to the same customer (as might be the
> case on your machine), you could set the cpu caps much higher, and depend
> on the shares for fairness.
>
> Examples for clarification:
>
> 1. I fire up a g4-highcpu-2G zone in the Joyent cloud. It gets a CPU cap
> of 100. It happens to land on a freshly deployed compute node with no other
> zones on it.
> I then run 2 CPU intensive processes. There is no contention so the
> machine is totally capable of scheduling each of them on a separate CPU and
> running them full throttle.
> However, because I'm only paying for 1 1vCPU I hit that cap of 100 and my
> two processes have to take turns running. If my workload is really that CPU
> intensive I should scale the zone up to g4-highcpu-4G which gets 2vCPUs and
> a cap of 200.
>
> 2. At home I have a machine with 8 cores running SmartOS. I don't put caps
> on my zones because I am the only customer on the box so to speak. I
> provision 3 zones on the machine: the "important" zone that gets 600
> shares, and two "unimportant" ones that each get 200 shares. Each of them
> is running 8 CPU intensive processes (apparently I'm a glutton for
> punishment). I have given out 1 shares. Because my numbers all divide
> evenly I can use simple fractions. The important zone get 6/10 or 3/5 of
> the CPU, and the other two each get 2/10 or 1/5. My math adds up: 3/5 + 1/5
> + 1/5 = 5/5 =1. The important zone gets approximately 4.8 cores worth of
> CPU time, and the unimportant ones each get approximately 1.6 cores worth.
> This is as good as things can be when I foolishly run too many CPU
> intensive workloads on my machine at home. Maybe it's time for a beefier
> box. :-P
>
> Was this helpful?
> -Nahum
>
>
Hi Nahum

Thats very useful. Inside the joyent public cloud how many cores do you see
inside an LX zone ? When I spin up an LZ zone I see all the cores on the
machine, which makes sense. So some customers see 40 Cores ("Intel(R)
Xeon(R) CPU E5-2670 v2 @ 2.50GHz") and some customers see 128 cores
("Intel(R) Xeon(R) CPU E7-4850 v4 @ 2.10GHz") regardless of the cpu_cap
setting.

The problem is as Ian said above that when cpu_cap is less than the core
count its tricky for our customers.

Thanks
Len



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Robert Mustacchi]

On 12/2/16 8:17 , Len Weincier wrote:
> On 2 December 2016 at 18:04, Robert Mustacchi  wrote:
> 
>> On 12/1/16 23:44 , Len Weincier wrote:

 This is the classic symptom of a machine with a cpu_cap less than its
>> CPU
 (core) count.

>>>
>>> So the vm sees all the cores and dispatches threads / processes for that
>>> many cores but the actual uasage is limited by the cpu_cap so the threads
>>> are stalling waiting for the OS to give them a slot to run hence the load
>>> avg climbing iiuc. Setting cpu_cap=0 sets the machine free and its
>>> responsive
>>>
>>> Whats the right way to share the cores for LX brand machines then seeing
>> as
>>> we cant limit the access to the cores ? In triton is see the default
>>> packages set the cpu_cap as well which would result in the same issue ?
>>
>> At the end of the day you need to make sure you actually tune things
>> inside of your zones to respect sizing based on what you've set in caps
>> and shares.
>>
>> Robert
>>
> 
> Hi Robert
> 
> Thanks for the reply.   These are machine we are provisioning for customers
> and we unfortunately have very little control over what customers are doing
> inside the zones. The ticket was logged by a customer saying that their app
> had ground to a halt after we moved them to the new hardware.
> 
> afaics this will happen with triton as well. We will be testing this in a
> lab to confirm.

It will. There's nothing in Triton that lies about the number of CPUs
available. Barring something like that, which is going to be very, very
hard to tell and not confuse things, the solution is tune things in
applications.

Most applications that tune based on visible CPUs aren't designed to
scale to the number you have with a 4 socket machine.

Robert


---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Len Weincier]

On 2 December 2016 at 18:04, Robert Mustacchi  wrote:

> On 12/1/16 23:44 , Len Weincier wrote:
> >>
> >> This is the classic symptom of a machine with a cpu_cap less than its
> CPU
> >> (core) count.
> >>
> >
> > So the vm sees all the cores and dispatches threads / processes for that
> > many cores but the actual uasage is limited by the cpu_cap so the threads
> > are stalling waiting for the OS to give them a slot to run hence the load
> > avg climbing iiuc. Setting cpu_cap=0 sets the machine free and its
> > responsive
> >
> > Whats the right way to share the cores for LX brand machines then seeing
> as
> > we cant limit the access to the cores ? In triton is see the default
> > packages set the cpu_cap as well which would result in the same issue ?
>
> At the end of the day you need to make sure you actually tune things
> inside of your zones to respect sizing based on what you've set in caps
> and shares.
>
> Robert
>

Hi Robert

Thanks for the reply.   These are machine we are provisioning for customers
and we unfortunately have very little control over what customers are doing
inside the zones. The ticket was logged by a customer saying that their app
had ground to a halt after we moved them to the new hardware.

afaics this will happen with triton as well. We will be testing this in a
lab to confirm.

Thanks
Len



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Robert Mustacchi]

On 12/1/16 23:44 , Len Weincier wrote:
>>
>> This is the classic symptom of a machine with a cpu_cap less than its CPU
>> (core) count.
>>
> 
> So the vm sees all the cores and dispatches threads / processes for that
> many cores but the actual uasage is limited by the cpu_cap so the threads
> are stalling waiting for the OS to give them a slot to run hence the load
> avg climbing iiuc. Setting cpu_cap=0 sets the machine free and its
> responsive
> 
> Whats the right way to share the cores for LX brand machines then seeing as
> we cant limit the access to the cores ? In triton is see the default
> packages set the cpu_cap as well which would result in the same issue ?

At the end of the day you need to make sure you actually tune things
inside of your zones to respect sizing based on what you've set in caps
and shares.

Robert




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Nahum Shalman]

I like to think about CPU as a resource either under contention (more
processes need CPU time than CPU cores to go around) or not:

CPU caps limit how much you can use when there's no contention. In the
Joyent Could it ensures people don't get more CPU time than they pay for.
(example 1 below)

CPU shares ensure fairness when there is contention. If CPU shares are
doled out in proportion to memory doled out (Like in the Joyent Cloud),
when everyone is fighting for CPU, the more memory you're paying for, the
more CPU cycles you'll get. (example 2 below)

If all the zones on a box belong to the same customer (as might be the case
on your machine), you could set the cpu caps much higher, and depend on the
shares for fairness.

Examples for clarification:

1. I fire up a g4-highcpu-2G zone in the Joyent cloud. It gets a CPU cap of
100. It happens to land on a freshly deployed compute node with no other
zones on it.
I then run 2 CPU intensive processes. There is no contention so the machine
is totally capable of scheduling each of them on a separate CPU and running
them full throttle.
However, because I'm only paying for 1 1vCPU I hit that cap of 100 and my
two processes have to take turns running. If my workload is really that CPU
intensive I should scale the zone up to g4-highcpu-4G which gets 2vCPUs and
a cap of 200.

2. At home I have a machine with 8 cores running SmartOS. I don't put caps
on my zones because I am the only customer on the box so to speak. I
provision 3 zones on the machine: the "important" zone that gets 600
shares, and two "unimportant" ones that each get 200 shares. Each of them
is running 8 CPU intensive processes (apparently I'm a glutton for
punishment). I have given out 1 shares. Because my numbers all divide
evenly I can use simple fractions. The important zone get 6/10 or 3/5 of
the CPU, and the other two each get 2/10 or 1/5. My math adds up: 3/5 + 1/5
+ 1/5 = 5/5 =1. The important zone gets approximately 4.8 cores worth of
CPU time, and the unimportant ones each get approximately 1.6 cores worth.
This is as good as things can be when I foolishly run too many CPU
intensive workloads on my machine at home. Maybe it's time for a beefier
box. :-P

Was this helpful?
-Nahum

On Fri, Dec 2, 2016 at 3:47 AM, Len Weincier  wrote:

> Just to note I see that in triton the cpu_cap is set as well as cpu_shares
> == max_physical_memory
>
> I am struggling to understand how this works. I am guessing that if the
> ratio of cpu_cap to cores is low then its not that noticeable to the
> guests, the threads get through in a reasonable amount of time but with the
> E7-x cpu's that have high core counts and the ratio is higher this will
> become a more serious problem especially for multithreaded apps ?
>
> Thanks
> Len
>
>
> On 2 December 2016 at 09:44, Len Weincier  wrote:
>
>> This is the classic symptom of a machine with a cpu_cap less than its
>>> CPU (core) count.
>>>
>>
>> So the vm sees all the cores and dispatches threads / processes for that
>> many cores but the actual uasage is limited by the cpu_cap so the threads
>> are stalling waiting for the OS to give them a slot to run hence the load
>> avg climbing iiuc. Setting cpu_cap=0 sets the machine free and its
>> responsive
>>
>> Whats the right way to share the cores for LX brand machines then seeing
>> as we cant limit the access to the cores ? In triton is see the default
>> packages set the cpu_cap as well which would result in the same issue ?
>>
>> I have had a look at https://wiki.smartos.org/di
>> splay/DOC/CPU+Caps+and+Shares  and am not sure what todo ?
>>
>> Thanks
>> Len
>>
>>
> *smartos-discuss* | Archives
> 
>  |
> Modify
> 
> Your Subscription 
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Len Weincier]

Just to note I see that in triton the cpu_cap is set as well as cpu_shares
== max_physical_memory

I am struggling to understand how this works. I am guessing that if the
ratio of cpu_cap to cores is low then its not that noticeable to the
guests, the threads get through in a reasonable amount of time but with the
E7-x cpu's that have high core counts and the ratio is higher this will
become a more serious problem especially for multithreaded apps ?

Thanks
Len


On 2 December 2016 at 09:44, Len Weincier  wrote:

> This is the classic symptom of a machine with a cpu_cap less than its CPU
>> (core) count.
>>
>
> So the vm sees all the cores and dispatches threads / processes for that
> many cores but the actual uasage is limited by the cpu_cap so the threads
> are stalling waiting for the OS to give them a slot to run hence the load
> avg climbing iiuc. Setting cpu_cap=0 sets the machine free and its
> responsive
>
> Whats the right way to share the cores for LX brand machines then seeing
> as we cant limit the access to the cores ? In triton is see the default
> packages set the cpu_cap as well which would result in the same issue ?
>
> I have had a look at https://wiki.smartos.org/display/DOC/CPU+Caps+and+
> Shares  and am not sure what todo ?
>
> Thanks
> Len
>
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Len Weincier]

>
> This is the classic symptom of a machine with a cpu_cap less than its CPU
> (core) count.
>

So the vm sees all the cores and dispatches threads / processes for that
many cores but the actual uasage is limited by the cpu_cap so the threads
are stalling waiting for the OS to give them a slot to run hence the load
avg climbing iiuc. Setting cpu_cap=0 sets the machine free and its
responsive

Whats the right way to share the cores for LX brand machines then seeing as
we cant limit the access to the cores ? In triton is see the default
packages set the cpu_cap as well which would result in the same issue ?

I have had a look at
https://wiki.smartos.org/display/DOC/CPU+Caps+and+Shares  and am not sure
what todo ?

Thanks
Len



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] New host and CPU question

[Ian Collins]

On 12/ 2/16 05:17 AM, Len Weincier wrote:

Hi all

Is there any issues with these cpu's and smartos - "Intel(R) Xeon(R) 
CPU E7-4850 v4 @ 2.10GHz" ?
We have a new host with 4 of those cpus and 1.5TB ram and moved some 
machines to the host which are now thrashing the cpu - load avg in a 
test zone with cpu_cap=800 is over 180 and just blocked the same vm on 
the old host was fine.


This is the classic symptom of a machine with a cpu_cap less than its 
CPU (core) count.


Cheers,

--
Ian.




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

[smartos-discuss] New host and CPU question

[Len Weincier]

Hi all

Is there any issues with these cpu's and smartos - "Intel(R) Xeon(R) CPU
E7-4850 v4 @ 2.10GHz" ?

We have a new host with 4 of those cpus and 1.5TB ram and moved some
machines to the host which are now thrashing the cpu - load avg in a test zone
with cpu_cap=800 is over 180 and just blocked the same vm on the old host
was fine.

The old host has "Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz" cpus.

Thanks
Len



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com