Re: [smartos-discuss] Re: New host and CPU question

[Casey Bisson]

To expand on Nigel’s excellent point, I’d suggest browsing through Intel and 
Facebook’s (independent, I think) research on scaling Memcached threads:

https://software.intel.com/en-us/articles/enhancing-the-scalability-of-memcached-0
 


https://www.facebook.com/notes/facebook-engineering/scaling-memcached-at-facebook/39391378919/
 


From Facebook’s work:

> memcached's stat collection relied on a global lock. A nuisance with 4 cores, 
> with 8 cores, the lock now accounted for 20-30% of CPU usage

That was a _while_ ago now, but it’s a pattern that we’ve seen repeated in a 
lot of apps (and I’m not even sure it’s fully fixed in Memcached today).

There are other reasons why scaling up thread count can counter-performant* for 
an app, don’t take my point to suggest locks are the only problem, just a solid 
example of one.

Where you can, you might want to configure the number of threads your app 
should attempt to start, separate from any heuristics it might have about CPU 
cores.

—Casey

* I hate saying “performant,” apologies.

> On Dec 1, 2016, at 9:26 AM, Nigel W  wrote:
> 
> Hello Len,
> 
> How many threads or processes is the workload generating?  Joyent and LX 
> zones see all cpus and memory of the host (regardless of cpu_cap setting as 
> far I know) so your workload may have gone overboard when it seen the 32*4 
> CPU threads and 1.5TB of RAM.  In particular, is the workload controlled by 
> configuration to a realistic number for having access to only 8 cpus of work?
> 
> Thanks,
> 
> On Thu, Dec 1, 2016 at 9:27 AM, Len Weincier  > wrote:
> Just to add to this. 
> 
> The issue is happening in an LX zone.
> SmartOS image is 20160915T211220Z
> 
> Thanks
> Len
> 
> On 1 December 2016 at 18:17, Len Weincier  > wrote:
> Hi all
> 
> Is there any issues with these cpu's and smartos - "Intel(R) Xeon(R) CPU 
> E7-4850 v4 @ 2.10GHz" ? 
> 
> We have a new host with 4 of those cpus and 1.5TB ram and moved some machines 
> to the host which are now thrashing the cpu - load avg in a test zone with 
> cpu_cap=800 is over 180 and just blocked the same vm on the old host was fine.
> 
> The old host has "Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz"  cpus.
> 
> Thanks
> Len
> 
> 
> smartos-discuss | Archives 
>   
>  | 
> Modify  Your Subscription   
> 



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] Re: New host and CPU question

[Nigel W]

Hello Len,

How many threads or processes is the workload generating?  Joyent and LX
zones see all cpus and memory of the host (regardless of cpu_cap setting as
far I know) so your workload may have gone overboard when it seen the 32*4
CPU threads and 1.5TB of RAM.  In particular, is the workload controlled by
configuration to a realistic number for having access to only 8 cpus of
work?

Thanks,

On Thu, Dec 1, 2016 at 9:27 AM, Len Weincier  wrote:

> Just to add to this.
>
> The issue is happening in an LX zone.
> SmartOS image is 20160915T211220Z
>
> Thanks
> Len
>
> On 1 December 2016 at 18:17, Len Weincier  wrote:
>
>> Hi all
>>
>> Is there any issues with these cpu's and smartos - "Intel(R) Xeon(R) CPU
>> E7-4850 v4 @ 2.10GHz" ?
>>
>> We have a new host with 4 of those cpus and 1.5TB ram and moved some
>> machines to the host which are now thrashing the cpu - load avg in a test 
>> zone
>> with cpu_cap=800 is over 180 and just blocked the same vm on the old
>> host was fine.
>>
>> The old host has "Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz" cpus.
>>
>> Thanks
>> Len
>>
>
> *smartos-discuss* | Archives
> 
>  |
> Modify
> 
> Your Subscription 
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com