Re: [Lxc-users] disk limit?
On 18.05.2011 20:59, Serge Hallyn wrote: Certainly not for loopback. Just make sure to create it as having a big hole in the middle, something like dd if=/dev/zero of=/srv/container1.rootfs.img bs=1M skip=1 count=1 Cool, I didn't know I can use sparse files for that. Good to know, thanks! However, if the container fills a 1G file and then deletes it, that space won't be freed. For LVM volumes, you can create one canonical container which takes up the space, then create other containers as snapshots of it. The snapshot containers won't take up space until the container starts changing blocks. That's a very interesting idea. I think it should provide very good performance too, because a lot of data/ pages will be shared between the different containers, right? I'll try that one, thanks again! :) Corin -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
On Wed, 18 May 2011, Serge Hallyn wrote: dd if=/dev/zero of=/srv/container1.rootfs.img bs=1M skip=1 count=1 That ought to be seek=1, not skip. (you skip the input, seek the outout) I'm not a fan of this though - if you create the image file(s) using dd there is a good chance it's going to be mostly consecutive blocks on the disk which is probably going to be more efficient in the long-run. Gordon -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
On 19.05.2011 09:59, Ulli Horlacher wrote: But how do you set up quotas for the snapshots? One can limit the size of the whole LVM container, but this is the same as using a regular disk partition (for all LXC containers). I'm by no means an lvm expert, but I would have guessed from Hallyn's suggestion that a writeable snapshot can not grow bigger than the underlying partition. So if you start with a base lvm partition of 10GB, each snapshot cannot take more than 10GB of modified data. So you end up with a rough 10GB quota per container. Corin -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
On Thu 2011-05-19 (10:35), Corin Langosch wrote: But how do you set up quotas for the snapshots? One can limit the size of the whole LVM container, but this is the same as using a regular disk partition (for all LXC containers). I'm by no means an lvm expert, but I would have guessed from Hallyn's suggestion that a writeable snapshot can not grow bigger than the underlying partition. But the underlaying partion must be big enouigh to contain all LXC containers! How do you prevent to a single container to allocate all free disk space? -- Ullrich Horlacher Server- und Arbeitsplatzsysteme Rechenzentrum E-Mail: horlac...@rus.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-685-65868 Allmandring 30 Fax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.rus.uni-stuttgart.de/ -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
On 19.05.2011 11:18, Ulli Horlacher wrote: But the underlaying partion must be big enouigh to contain all LXC containers! How do you prevent to a single container to allocate all free disk space? I had no time to consult the man pages or to just give it try. Have you tried it? But I guess each snapshot can grow to the maximum size of the underlying volume, no matter how big other snapshots have grown. Initial setup: disc: 500 GB (one big lvm partition) lvm volume: 10 GB (has vserver base system installation) = free space on disk: 490 GB Now you can create snapshots of this lv volume. When just created they will have no disk usage at all. disc: 500 GB (one big lvm partition) lvm volume: 10 GB (has vserver base system installation) snapshot 1: 0 GB (no individual data written so far) snapshot 2: 0 GB (no individual data written so far) snapshot 3: 0 GB (no individual data written so far) = free space on disk: 490 GB After some time users install data on their vservers and so the snapshots grow over time. disc: 500 GB (one big lvm partition) lvm volume: 10 GB (has vserver base system installation) snapshot 1: 5 GB (a lot of individual data written so far) snapshot 2: 10 GB (ups, no space left on device) snapshot 3: 1 GB (not so much individual data written so far) = free space on disk: 474 GB Otherwise Serge's suggestion wouldn't make any sense to me. Corin -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
I've used ZFS on Fuse before, with OpenVZ. The performance was horrible, but the flexibility outweighs the cons for small setups. No space was lost, thanks to volume management as well as deduplication support. For serious setups, I'd recommend exporting ZFS over NFS from a Nexenta host (or FreeBSD perhaps) so that it's reasonably fast. ACL support is also best done on NFS (note how ZFS uses NFS4 ACL scheme). In a homogenous environment, btrfs might provide a similar set of benefits, but I've hardly done any tests. Hope that helps. Robert Kawecki - Reply message - Od: Ulli Horlacher frams...@rus.uni-stuttgart.de Do: lxc-users@lists.sourceforge.net Temat: [Lxc-users] disk limit? Data: śr., maj 18, 2011 17:30 Is there an easy way to set up a disk limit for a container? I could create a LVM partition for each container, but this is not what I call easy :-} -- Ullrich Horlacher Server- und Arbeitsplatzsysteme Rechenzentrum E-Mail: horlac...@rus.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-685-65868 Allmandring 30 Fax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.rus.uni-stuttgart.de/ -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
Quoting Corin Langosch (cor...@gmx.de): On 19.05.2011 11:18, Ulli Horlacher wrote: After some time users install data on their vservers and so the snapshots grow over time. disc: 500 GB (one big lvm partition) lvm volume: 10 GB (has vserver base system installation) snapshot 1: 5 GB (a lot of individual data written so far) snapshot 2: 10 GB (ups, no space left on device) snapshot 3: 1 GB (not so much individual data written so far) = free space on disk: 474 GB Otherwise Serge's suggestion wouldn't make any sense to me. Right - it'll let you overcommit like mad to create the containers to begin with. But it won't enforce the limit. You can use a script on the host to watch the actual usage and kindly ask the users to be careful. I've tried enforcing a smaller limit by doing lvcreate -L 2G -n delme1 lxc mkfs.xfs /dev/lxc/delme1 lvcreate -s /dev/lxc/delme1 -L 100M -n delme2 but /dev/lxc/delme2 does not get a 100M limit, unfortunately. -serge -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
and what about using xfs quota by project? is somebody tried? On Thu, May 19, 2011 at 8:04 AM, Serge Hallyn serge.hal...@canonical.com wrote: Quoting Corin Langosch (cor...@gmx.de): On 19.05.2011 11:18, Ulli Horlacher wrote: After some time users install data on their vservers and so the snapshots grow over time. disc: 500 GB (one big lvm partition) lvm volume: 10 GB (has vserver base system installation) snapshot 1: 5 GB (a lot of individual data written so far) snapshot 2: 10 GB (ups, no space left on device) snapshot 3: 1 GB (not so much individual data written so far) = free space on disk: 474 GB Otherwise Serge's suggestion wouldn't make any sense to me. Right - it'll let you overcommit like mad to create the containers to begin with. But it won't enforce the limit. You can use a script on the host to watch the actual usage and kindly ask the users to be careful. I've tried enforcing a smaller limit by doing lvcreate -L 2G -n delme1 lxc mkfs.xfs /dev/lxc/delme1 lvcreate -s /dev/lxc/delme1 -L 100M -n delme2 but /dev/lxc/delme2 does not get a 100M limit, unfortunately. -serge -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- François-Xavier Bourlet -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
Quoting Ulli Horlacher (frams...@rus.uni-stuttgart.de): Is there an easy way to set up a disk limit for a container? I could create a LVM partition for each container, but this is not what I call easy :-} (Not trying to argue, just probe) Why do you call it not easy? Because you don't have spare partitions to dedicate to a pv? Or because you're not used to using lvm? If the former, then you could use a loopback filesystem instead of an LVM. I assume that'll impact performance, but I've not tested it to see by how much. If the latter, then in the next few months I intend to push some stuff to lxc to integrate LVM usage. Daniel had had comments to my first patches so it'll likely change, but what I'm using right now let's me just do lxc-lvmcreate in place of lxc-create to create a lvm-backed lxc partition, and 'lxc-clone -s -o c1 -n c2' lets me create container c2 with a lvm snapshot of c1's rootfs. (See http://s3hh.wordpress.com/2011/03/30/lxc-lvm-clone/ and http://s3hh.wordpress.com/2011/03/30/one-more-lxc-clone-update/) There's no cgroup to do what you want, though. -serge -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] disk limit?
On 18.05.2011 17:52, Serge Hallyn wrote: Why do you call it not easy? Because you don't have spare partitions to dedicate to a pv? Or because you're not used to using lvm? If the former, then you could use a loopback filesystem instead of an LVM. I assume that'll impact performance, but I've not tested it to see by how much. If the latter, then in the next few months I intend to push some stuff to lxc to integrate LVM usage. Daniel had had comments to my first patches so it'll likely change, but what I'm using right now let's me just do lxc-lvmcreate in place of lxc-create to create a lvm-backed lxc partition, and 'lxc-clone -s -o c1 -n c2' lets me create container c2 with a lvm snapshot of c1's rootfs. (See http://s3hh.wordpress.com/2011/03/30/lxc-lvm-clone/ and http://s3hh.wordpress.com/2011/03/30/one-more-lxc-clone-update/) There's no cgroup to do what you want, though. I might be wrong, but I think the biggest disadvantage (show-stopper) of lvm/ loopback is that the partition/ image will consume the whole space even when not a single file is actually stored in the fs. For example imagine you have a 500 GB hardisk and want to create 50 vservers with an 50 GB diskspace limit each. This is not possible with lvm or loopback devices because one would need 2500 GB storage. But it's a very common use case because in average only 5% of the vservers will actually use 50GB and so you'll never run out of space if the space would be allocated on demand. Corin -- What Every C/C++ and Fortran developer Should Know! Read this article and learn how Intel has extended the reach of its next-generation tools to help Windows* and Linux* C/C++ and Fortran developers boost performance applications - including clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
