Re: [lxc-users] LXD and Kernel Samepage Merging (KSM)

[Fajar A. Nugraha]

On Mon, Jun 5, 2017 at 9:02 AM, Ron Kelley  wrote:

> Thanks for the feedback and info.  Seems something is amiss with my setup.
>
> What does your Ubuntu install look like?  I have a standard Ubuntu 16.04
> with up-to-date patches and with LXD 2.12.  Guess I need to find the
> differences between your setup and mine.
>
>
 # cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"

# dpkg -l | grep linux-image
ii  linux-image-4.10.0-21-generic4.10.0-21.23~16.04.1
amd64Linux kernel image for version 4.10.0 on 64 bit x86 SMP
ii  linux-image-virtual-hwe-16.04-edge   4.10.0.21.14
amd64Virtual Linux kernel image

# dpkg -l | grep lxd
ii  lxd  2.13-0ubuntu3~ubuntu16.04.1
 amd64Container hypervisor based on LXC - daemon
ii  lxd-client   2.13-0ubuntu3~ubuntu16.04.1
 amd64Container hypervisor based on LXC - client

-- 
Fajar
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] LXD and Kernel Samepage Merging (KSM)

[Ron Kelley]

Thanks for the feedback and info.  Seems something is amiss with my setup.

What does your Ubuntu install look like?  I have a standard Ubuntu 16.04 with 
up-to-date patches and with LXD 2.12.  Guess I need to find the differences 
between your setup and mine.

Thanks for any help!

-Ron



> On Jun 4, 2017, at 9:58 PM, Fajar A. Nugraha  wrote:
> 
> On Mon, Jun 5, 2017 at 7:48 AM, Ron Kelley  wrote:
> 
> As for the openvz link; I read that a few times but I don’t get any positive 
> results using those methods.  This leads me to believe (a) LXD does not 
> support KSM or (b) the applications are not registering w/the KSM part of the 
> kernel.
> 
> 
> 
> Works for me here on a simple test, in 2 privileged containers.
> 
> # grep -H '' /sys/kernel/mm/ksm/*
> /sys/kernel/mm/ksm/full_scans:4488
> /sys/kernel/mm/ksm/merge_across_nodes:1
> /sys/kernel/mm/ksm/pages_shared:3
> /sys/kernel/mm/ksm/pages_sharing:3
> /sys/kernel/mm/ksm/pages_to_scan:100
> /sys/kernel/mm/ksm/pages_unshared:18
> /sys/kernel/mm/ksm/pages_volatile:0
> /sys/kernel/mm/ksm/run:1
> /sys/kernel/mm/ksm/sleep_millisecs:20
> /sys/kernel/mm/ksm/use_zero_pages:0
> 
> - build ksm-wrapper from source
> - patch ksm-wrapper as mentioned on 
> https://github.com/unbrice/ksm_preload/issues/6
> - on host, run 'echo 1 > /sys/kernel/mm/ksm/run'
> - on container 1 & 2 , run 'ksm-wrapper bash', and the (on both bash prompt) 
> run htop
> 
> pages_shared and pages_sharing will have non-zero value when htop is running 
> on both containers
> 
> -- 
> Fajar
> 
> ___
> lxc-users mailing list
> lxc-users@lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users

___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] LXD and Kernel Samepage Merging (KSM)

[Fajar A. Nugraha]

On Mon, Jun 5, 2017 at 7:48 AM, Ron Kelley  wrote:

>
> As for the openvz link; I read that a few times but I don’t get any
> positive results using those methods.  This leads me to believe (a) LXD
> does not support KSM or (b) the applications are not registering w/the KSM
> part of the kernel.
>
>

Works for me here on a simple test, in 2 privileged containers.

# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:4488
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:3
/sys/kernel/mm/ksm/pages_sharing:3
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:18
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:1
/sys/kernel/mm/ksm/sleep_millisecs:20
/sys/kernel/mm/ksm/use_zero_pages:0

- build ksm-wrapper from source
- patch ksm-wrapper as mentioned on
https://github.com/unbrice/ksm_preload/issues/6
- on host, run 'echo 1 > /sys/kernel/mm/ksm/run'
- on container 1 & 2 , run 'ksm-wrapper bash', and the (on both bash
prompt) run htop

pages_shared and pages_sharing will have non-zero value when htop is
running on both containers

-- 
Fajar
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] LXD and Kernel Samepage Merging (KSM)

[Ron Kelley]

Thanks Fajar. 

This is on-site with our own physical servers, storage, etc.  The goal is to 
get the most containers per server as possible.  While our servers have lots of 
RAM, we need to come up with a long-term scaling plan and hope KSM can help us 
scale beyond the standard numbers.

As for the openvz link; I read that a few times but I don’t get any positive 
results using those methods.  This leads me to believe (a) LXD does not support 
KSM or (b) the applications are not registering w/the KSM part of the kernel.

I am going to run through some tests this week to see if I can get KSM working 
outside the LXD environment then try to replicate the same tests inside LXD.

Thanks again for the feedback.



> On Jun 4, 2017, at 6:15 PM, Fajar A. Nugraha  wrote:
> 
> On Sun, Jun 4, 2017 at 11:16 PM, Ron Kelley  wrote:
> (Reviving the thread about Container Scaling:  
> https://lists.linuxcontainers.org/pipermail/lxc-users/2016-May/011607.html)
> 
> We have hit critical mass with LXD 2.12 and I need to get Kernel Samepage 
> Merging (KSM) working as soon as possible.  All my research has come to a 
> dead-end, and I am reaching out to the group at large for suggestions.
> 
> Background: We have 5 host servers - each running U16.04 (4.4.0-57-generic), 
> 8G RAM, 20G SWAP, and 50 containers (exact configs per server - nginx and php 
> 7).
> 
> 
> Is this a cloud, or on-site setup?
> 
> For cloud, there are a lot of options that could get you running with MUCH 
> more memory, which would save you lots of headaches getting KSM to work. My 
> favorite is EC2 spot instance on AWS.
> 
> On another note, I now setup most of my hosts with no swap, since performance 
> plummets whenever swap is used. YMMV.
> 
>  
> I am trying to get KSM working since each container is an identical replica 
> of the other (other than hostname/IP).  I have read a ton of information on 
> the ‘net about Ubuntu and KSM, yet I can’t seem to get any pages to share on 
> the host.  I am not sure if this is a KSM config issue or if LXD won’t allow 
> KSM between containers.
> 
> 
> 
>  
> 
> Here is what I have done thus far:
> --
> * Installed the ksmtuned utility and verified ksmd is running on each host.
> * Created the ksm_preload and ksm-wrapper tools per this site (the 
> https://github.com/unbrice/ksm_preload).
> * Created 50 identical Ubuntu 16.04 containers running nginx
> * Modified the nginx startup script on each container to include the 
> ksm_preload.so library; no issues running nginx.
> 
> (Note: since I could not find the ksm_preload library for Ubuntu, I had to 
> use the ksm-wrapper tool listed above)
> 
> All the relevant files under /sys/kernel/mm/ksm still show 0 (pages_shared, 
> pages_sharing, etc) regardless of what I do.
> 
> 
> Can any (@stgraber @brauner) confirm if KSM is supported with LXD?  If so, 
> what is the “magic” to make it work?  We really want to get 2-3x more sites 
> per container if possible.
> 
> 
> Have you read https://openvz.org/KSM_(kernel_same-page_merging) ? Some info 
> might be relevant. For example, it mentions something which you did not wrote:
> 
> To start ksmd, issue
> [root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run
> 
> Also the section about Tuning and Caveats.
> 
> -- 
> Fajar
> ___
> lxc-users mailing list
> lxc-users@lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users

___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] LXD and Kernel Samepage Merging (KSM)

[Fajar A. Nugraha]

On Sun, Jun 4, 2017 at 11:16 PM, Ron Kelley  wrote:

> (Reviving the thread about Container Scaling:
> https://lists.linuxcontainers.org/pipermail/lxc-users/2016-May/011607.html
> )
>
> We have hit critical mass with LXD 2.12 and I need to get Kernel Samepage
> Merging (KSM) working as soon as possible.  All my research has come to a
> dead-end, and I am reaching out to the group at large for suggestions.
>
> Background: We have 5 host servers - each running U16.04
> (4.4.0-57-generic), 8G RAM, 20G SWAP, and 50 containers (exact configs per
> server - nginx and php 7).
>
>
Is this a cloud, or on-site setup?

For cloud, there are a lot of options that could get you running with MUCH
more memory, which would save you lots of headaches getting KSM to work. My
favorite is EC2 spot instance on AWS.

On another note, I now setup most of my hosts with no swap, since
performance plummets whenever swap is used. YMMV.



> I am trying to get KSM working since each container is an identical
> replica of the other (other than hostname/IP).  I have read a ton of
> information on the ‘net about Ubuntu and KSM, yet I can’t seem to get any
> pages to share on the host.  I am not sure if this is a KSM config issue or
> if LXD won’t allow KSM between containers.
>
>



>
> Here is what I have done thus far:
> --
> * Installed the ksmtuned utility and verified ksmd is running on each host.
> * Created the ksm_preload and ksm-wrapper tools per this site (the
> https://github.com/unbrice/ksm_preload).
> * Created 50 identical Ubuntu 16.04 containers running nginx
> * Modified the nginx startup script on each container to include the
> ksm_preload.so library; no issues running nginx.
>
> (Note: since I could not find the ksm_preload library for Ubuntu, I had to
> use the ksm-wrapper tool listed above)
>
> All the relevant files under /sys/kernel/mm/ksm still show 0
> (pages_shared, pages_sharing, etc) regardless of what I do.
>
>
> Can any (@stgraber @brauner) confirm if KSM is supported with LXD?  If so,
> what is the “magic” to make it work?  We really want to get 2-3x more sites
> per container if possible.
>
>
Have you read https://openvz.org/KSM_(kernel_same-page_merging) ? Some info
might be relevant. For example, it mentions something which you did not
wrote:

To start ksmd, issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

Also the section about Tuning and Caveats.

-- 
Fajar
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] LXD and Kernel Samepage Merging (KSM)

[Stéphane Graber]

On Sun, Jun 04, 2017 at 12:16:50PM -0400, Ron Kelley wrote:
> (Reviving the thread about Container Scaling:  
> https://lists.linuxcontainers.org/pipermail/lxc-users/2016-May/011607.html)
> 
> We have hit critical mass with LXD 2.12 and I need to get Kernel Samepage 
> Merging (KSM) working as soon as possible.  All my research has come to a 
> dead-end, and I am reaching out to the group at large for suggestions.
> 
> Background: We have 5 host servers - each running U16.04 (4.4.0-57-generic), 
> 8G RAM, 20G SWAP, and 50 containers (exact configs per server - nginx and php 
> 7).
> 
> I am trying to get KSM working since each container is an identical replica 
> of the other (other than hostname/IP).  I have read a ton of information on 
> the ‘net about Ubuntu and KSM, yet I can’t seem to get any pages to share on 
> the host.  I am not sure if this is a KSM config issue or if LXD won’t allow 
> KSM between containers.
> 
> 
> Here is what I have done thus far:
> --
> * Installed the ksmtuned utility and verified ksmd is running on each host.
> * Created the ksm_preload and ksm-wrapper tools per this site (the 
> https://github.com/unbrice/ksm_preload).
> * Created 50 identical Ubuntu 16.04 containers running nginx
> * Modified the nginx startup script on each container to include the 
> ksm_preload.so library; no issues running nginx.
> 
> (Note: since I could not find the ksm_preload library for Ubuntu, I had to 
> use the ksm-wrapper tool listed above)
> 
> All the relevant files under /sys/kernel/mm/ksm still show 0 (pages_shared, 
> pages_sharing, etc) regardless of what I do.
> 
> 
> Can any (@stgraber @brauner) confirm if KSM is supported with LXD?  If so, 
> what is the “magic” to make it work?  We really want to get 2-3x more sites 
> per container if possible.
> 
> 
> Thanks.

I'm not familiar with the use of KSM outside of the typicaly virtual
machine case (where a single process, "qemu", own the whole VM memory
and can properly mark things for KSM to merge).

LXD doesn't own or control the memory of its containers, so I don't
think there's anything that LXD itself can do to allow or prevent the
use of KSM.

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com


signature.asc
Description: PGP signature
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

[lxc-users] LXD and Kernel Samepage Merging (KSM)

[Ron Kelley]

(Reviving the thread about Container Scaling:  
https://lists.linuxcontainers.org/pipermail/lxc-users/2016-May/011607.html)

We have hit critical mass with LXD 2.12 and I need to get Kernel Samepage 
Merging (KSM) working as soon as possible.  All my research has come to a 
dead-end, and I am reaching out to the group at large for suggestions.

Background: We have 5 host servers - each running U16.04 (4.4.0-57-generic), 8G 
RAM, 20G SWAP, and 50 containers (exact configs per server - nginx and php 7).

I am trying to get KSM working since each container is an identical replica of 
the other (other than hostname/IP).  I have read a ton of information on the 
‘net about Ubuntu and KSM, yet I can’t seem to get any pages to share on the 
host.  I am not sure if this is a KSM config issue or if LXD won’t allow KSM 
between containers.


Here is what I have done thus far:
--
* Installed the ksmtuned utility and verified ksmd is running on each host.
* Created the ksm_preload and ksm-wrapper tools per this site (the 
https://github.com/unbrice/ksm_preload).
* Created 50 identical Ubuntu 16.04 containers running nginx
* Modified the nginx startup script on each container to include the 
ksm_preload.so library; no issues running nginx.

(Note: since I could not find the ksm_preload library for Ubuntu, I had to use 
the ksm-wrapper tool listed above)

All the relevant files under /sys/kernel/mm/ksm still show 0 (pages_shared, 
pages_sharing, etc) regardless of what I do.


Can any (@stgraber @brauner) confirm if KSM is supported with LXD?  If so, what 
is the “magic” to make it work?  We really want to get 2-3x more sites per 
container if possible.


Thanks.

___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users