Re: [one-users] Thin provisioning and image size
Interestingly, I found that I was able to do some overcommitting after all, somewhat to my surprise, as my earlier tests of open nebula 4.4 and 4.6 didn't seem to allow that. This is what I think is happening: Beginning setup: Image datastore is on NFS, system datastore is on local disk, launching non-persistent image via CLONE operation in opennebula. DATASTORE_CAPACITY_CHECK = NO for both. Initial size of system data store, 450 GB. Max size of qcow2 image--262 GB Actuall size of qcow2 image on disk, unexpanded-2.6 GB Launch first VM--SCHED asks--does system datastore have 262 GB available? Yes. Launch second VM--SCHED asks--does system datastore have 262 GB available? Yes.. because first one hasn't expanded out. and so forth. That amount of overprovisioning is enough for our purposes--we can get 8-9 images (actually a lot more than that) into the datastore and will only get in trouble if our users all run out the file to full size, which most of them do not. The system runs out of RAM and CPU before it runs out of disk in this scenario. Steve Timm From: Ruben S. Montero [rube...@dacya.ucm.es] Sent: Wednesday, October 08, 2014 3:50 AM To: Steven C Timm Cc: Javier Fontan; users@lists.opennebula.org Subject: Re: [one-users] Thin provisioning and image size Hi Yes, the value for the size of the image is computed in libfs.sh (fs_size) for different file types. For qcow2 images: SIZE=$($QEMU_IMG info $1 | sed -n 's/.*(\([0-9]*\) bytes).*/\1/p') I.e. the virtual size in bytes is used. As this is the max. value of the image it is not updated. Some thoughts: 1.- Using the max size, enables a proper enforcement of disk quotas. And so prevents a kind of DoS attack , when a user starts a VM and grows the disk over the size of the Datastore, effectively disabling the hypervisor. 2.- Initially we thought of updating the image size every time it is copied back to the image datastore, but this may conflict with the quota system, as mentioned above. 3.- If needed we could implement datastore over-commitment as we do with the hosts, i.e. extend the LIMIT_MB attribute to allow OpenNebula use more datastore storage. (See, http://docs.opennebula.org/4.8/administration/references/ds_conf.html) Cheers Ruben On Mon, Oct 6, 2014 at 7:45 PM, Steven Timm t...@fnal.govmailto:t...@fnal.gov wrote: I am seeing the opposite problem in One 4.x and have been ever since we started testing it. When I do oneimage create using a qcow2 image, opennebula always reports the size as the absolute maximum to which the qcow2 file system could expand. This keeps us from being able to over-provision our disk on the VM hosts as we've done under Opennebula 3.2 for a long time. For instance: [oneadmin@fermicloud198 ~]$ oneimage show 5 IMAGE 5 INFORMATION ID : 5 NAME : SLF6Vanilla USER : oneadmin GROUP : oneadmin DATASTORE : cloud_images TYPE : OS REGISTER TIME : 10/03 16:31:31 PERSISTENT : No SOURCE : /var/lib/one/datastores/102/180caf99a13146dbd1b60593378d4479 PATH : /tmp/55c42a4cc7f87ea3390bc2bef14212c5 SIZE : 256G STATE : used RUNNING_VMS: 1 PERMISSIONS OWNER : um- GROUP : u-- OTHER : u-- IMAGE TEMPLATE DESCRIPTION=SLF6 Vanilla DEV_PREFIX=vd DRIVER=qcow2 EC2_AMI=YES [oneadmin@fermicloud198 ~]$ onedatastore show 102 DATASTORE 102 INFORMATION ID : 102 NAME : cloud_images USER : njp GROUP : oneadmin CLUSTER: cloudworker TYPE : IMAGE DS_MAD : fs TM_MAD : shared BASE PATH : /var/lib/one/datastores/102 DISK_TYPE : FILE DATASTORE CAPACITY TOTAL: : 7T FREE: : 1.6T USED: : 4.1G LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE BASE_PATH=/var/lib/one/datastores/ CLONE_TARGET=SYSTEM DATASTORE_CAPACITY_CHECK=NO DISK_TYPE=FILE DS_MAD=fs LN_TARGET=NONE TM_MAD=shared TYPE=IMAGE_DS IMAGES 5 6 [oneadmin@fermicloud198 ~]$ onedatastore show 100 DATASTORE 100 INFORMATION ID : 100 NAME : localnode USER : oneadmin GROUP : oneadmin CLUSTER: cloudworker TYPE : SYSTEM DS_MAD : - TM_MAD : ssh BASE PATH : /var/lib/one/datastores/100/100 DISK_TYPE : FILE DATASTORE CAPACITY TOTAL: : - FREE: : - USED: : - LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE BASE_PATH=/var/lib/one/datastores/100/ DATASTORE_CAPACITY_CHECK=no SHARED=NO TM_MAD=ssh TYPE=SYSTEM_DS IMAGES [oneadmin@fermicloud198 ~]$ - Any suggestions? Steve On Mon, 8 Sep 2014, Javier Fontan wrote: Then the value makes sense as the units stored are Megabytes. On Fri, Sep 5, 2014 at 3:34 PM, Daniel
Re: [one-users] Thin provisioning and image size
I am seeing the opposite problem in One 4.x and have been ever since we started testing it. When I do oneimage create using a qcow2 image, opennebula always reports the size as the absolute maximum to which the qcow2 file system could expand. This keeps us from being able to over-provision our disk on the VM hosts as we've done under Opennebula 3.2 for a long time. For instance: [oneadmin@fermicloud198 ~]$ oneimage show 5 IMAGE 5 INFORMATION ID : 5 NAME : SLF6Vanilla USER : oneadmin GROUP : oneadmin DATASTORE : cloud_images TYPE : OS REGISTER TIME : 10/03 16:31:31 PERSISTENT : No SOURCE : /var/lib/one/datastores/102/180caf99a13146dbd1b60593378d4479 PATH : /tmp/55c42a4cc7f87ea3390bc2bef14212c5 SIZE : 256G STATE : used RUNNING_VMS: 1 PERMISSIONS OWNER : um- GROUP : u-- OTHER : u-- IMAGE TEMPLATE DESCRIPTION=SLF6 Vanilla DEV_PREFIX=vd DRIVER=qcow2 EC2_AMI=YES [oneadmin@fermicloud198 ~]$ onedatastore show 102 DATASTORE 102 INFORMATION ID : 102 NAME : cloud_images USER : njp GROUP : oneadmin CLUSTER: cloudworker TYPE : IMAGE DS_MAD : fs TM_MAD : shared BASE PATH : /var/lib/one/datastores/102 DISK_TYPE : FILE DATASTORE CAPACITY TOTAL: : 7T FREE: : 1.6T USED: : 4.1G LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE BASE_PATH=/var/lib/one/datastores/ CLONE_TARGET=SYSTEM DATASTORE_CAPACITY_CHECK=NO DISK_TYPE=FILE DS_MAD=fs LN_TARGET=NONE TM_MAD=shared TYPE=IMAGE_DS IMAGES 5 6 [oneadmin@fermicloud198 ~]$ onedatastore show 100 DATASTORE 100 INFORMATION ID : 100 NAME : localnode USER : oneadmin GROUP : oneadmin CLUSTER: cloudworker TYPE : SYSTEM DS_MAD : - TM_MAD : ssh BASE PATH : /var/lib/one/datastores/100/100 DISK_TYPE : FILE DATASTORE CAPACITY TOTAL: : - FREE: : - USED: : - LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE BASE_PATH=/var/lib/one/datastores/100/ DATASTORE_CAPACITY_CHECK=no SHARED=NO TM_MAD=ssh TYPE=SYSTEM_DS IMAGES [oneadmin@fermicloud198 ~]$ - Any suggestions? Steve On Mon, 8 Sep 2014, Javier Fontan wrote: Then the value makes sense as the units stored are Megabytes. On Fri, Sep 5, 2014 at 3:34 PM, Daniel Dehennin daniel.dehen...@baby-gnu.org wrote: Javier Fontan jfon...@opennebula.org writes: Which was the size of the original image? I think that when you do a save_as (deferred disk snapshot) it just copies the size of the original image to the new one. I started with empty qcow2 disk of several virtual sizes, but on disk they are all 196KB. Regards. -- Daniel Dehennin Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org -- Javier Fontán Muiños Developer OpenNebula - Flexible Enterprise Cloud Made Simple www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org -- Steven C. Timm, Ph.D (630) 840-8525 t...@fnal.gov http://home.fnal.gov/~timm/ Fermilab Scientific Computing Division, Scientific Computing Services Quad. Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Thin provisioning and image size
Then the value makes sense as the units stored are Megabytes. On Fri, Sep 5, 2014 at 3:34 PM, Daniel Dehennin daniel.dehen...@baby-gnu.org wrote: Javier Fontan jfon...@opennebula.org writes: Which was the size of the original image? I think that when you do a save_as (deferred disk snapshot) it just copies the size of the original image to the new one. I started with empty qcow2 disk of several virtual sizes, but on disk they are all 196KB. Regards. -- Daniel Dehennin Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org -- Javier Fontán Muiños Developer OpenNebula - Flexible Enterprise Cloud Made Simple www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Thin provisioning and image size
Which was the size of the original image? I think that when you do a save_as (deferred disk snapshot) it just copies the size of the original image to the new one. On Tue, Sep 2, 2014 at 2:09 PM, Daniel Dehennin daniel.dehen...@baby-gnu.org wrote: Hello, The size of my qcow2 images is always reported as 1M: IMAGE 114 INFORMATION ID : 114 NAME : debian-wheezy USER : nebula GROUP : nebula DATASTORE : default TYPE : OS REGISTER TIME : 04/10 10:18:26 PERSISTENT : No SOURCE : /var/lib/one/datastores/1/5d2be6f9746fc3447be11c6621da87df FSTYPE : save_as SIZE : 1M STATE : rdy RUNNING_VMS: 0 PERMISSIONS OWNER : um- GROUP : u-- OTHER : u-- IMAGE TEMPLATE DEV_PREFIX=vd DRIVER=qcow2 SAVED_DISK_ID=0 SAVED_IMAGE_ID=17 SAVED_VM_ID=347 SAVE_AS=YES As far as I understand, the size is calculated only at creation time and never updated. I think we could differentiate the real size from the virtual size in case of thin provisioning like qcow2, qemu-img actually reports: qemu-img info /var/lib/one/datastores/1/5d2be6f9746fc3447be11c6621da87df image: /var/lib/one/datastores/1/5d2be6f9746fc3447be11c6621da87df file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 629M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false Regards. -- Daniel Dehennin Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org -- Javier Fontán Muiños Developer OpenNebula - Flexible Enterprise Cloud Made Simple www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Thin provisioning and image size
Javier Fontan jfon...@opennebula.org writes: Which was the size of the original image? I think that when you do a save_as (deferred disk snapshot) it just copies the size of the original image to the new one. I started with empty qcow2 disk of several virtual sizes, but on disk they are all 196KB. Regards. -- Daniel Dehennin Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
[one-users] Thin provisioning and image size
Hello, The size of my qcow2 images is always reported as 1M: IMAGE 114 INFORMATION ID : 114 NAME : debian-wheezy USER : nebula GROUP : nebula DATASTORE : default TYPE : OS REGISTER TIME : 04/10 10:18:26 PERSISTENT : No SOURCE : /var/lib/one/datastores/1/5d2be6f9746fc3447be11c6621da87df FSTYPE : save_as SIZE : 1M STATE : rdy RUNNING_VMS: 0 PERMISSIONS OWNER : um- GROUP : u-- OTHER : u-- IMAGE TEMPLATE DEV_PREFIX=vd DRIVER=qcow2 SAVED_DISK_ID=0 SAVED_IMAGE_ID=17 SAVED_VM_ID=347 SAVE_AS=YES As far as I understand, the size is calculated only at creation time and never updated. I think we could differentiate the real size from the virtual size in case of thin provisioning like qcow2, qemu-img actually reports: qemu-img info /var/lib/one/datastores/1/5d2be6f9746fc3447be11c6621da87df image: /var/lib/one/datastores/1/5d2be6f9746fc3447be11c6621da87df file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 629M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false Regards. -- Daniel Dehennin Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF signature.asc Description: PGP signature ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
