[vdsm] Exploiting domain specific offload features
Hello, We are adding features such as server offloaded cloning, snapshot of the files(ie VM disks) and zeroed vm disk allocation in GlusterFS. As of now only BD xlator supports offloaded cloning snapshot. Server offloaded zeroing of VM disks is supported both by posix and BD xlator. The initial approach is to use xattr interface to trigger this offload features such as # setfattr -n clone -v path-to-new-clone-file path-to-source-file will create clone of path-to-source-file in path-to-new-clone-file. Cloning is done in GlusterFS server side and its kind of server offloaded copy. Similarly snapshot can also be taken using setfattr approach. GlusterFS storage domain is already part of VDSM and we want to exploit offload features provided by GlusterFS through VDSM. Is there any way to exploit these features from VDSM as of now? ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Exploiting domain specific offload features
On 07/17/2013 12:48 PM, M. Mohan Kumar wrote: Hello, We are adding features such as server offloaded cloning, snapshot of the files(ie VM disks) and zeroed vm disk allocation in GlusterFS. As of now only BD xlator supports offloaded cloning snapshot. Server offloaded zeroing of VM disks is supported both by posix and BD xlator. The initial approach is to use xattr interface to trigger this offload features such as # setfattr -n clone -v path-to-new-clone-file path-to-source-file will create clone of path-to-source-file in path-to-new-clone-file. Cloning is done in GlusterFS server side and its kind of server offloaded copy. Similarly snapshot can also be taken using setfattr approach. GlusterFS storage domain is already part of VDSM and we want to exploit offload features provided by GlusterFS through VDSM. Is there any way to exploit these features from VDSM as of now? Mohan, IIUC, zeroing of files in GlusterFS is supported for both posix and block backends of GlusterFS Today VDSM does zeroing (as part of preallocated vmdisk flow) itself using 'dd'. If GlusterFS supports zeroing this feature can be exploited in VDSM (by overriding create volume flow as needed) so that we can save compute resources on the VDSM host, when Gluster domain is being used. Regarding exploiting clone and snapshot, IIUC these are very native to VDSM today... it expects that snapshots are qcow2 based and they form the image chain etc... With snapshot and clone handled in Gluster transparently, these notions of VDSM will be broken, so it probably needs a lot of changes in lots of places in VDSM to exploit these. Federico/Ayal, Wanted to know your comments/opinion on this ? Is there a way to exploit these features in VDSM Gluster storage domain in an elegant way ? thanx, deepak ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
- Original Message - | From: Michal Skrivanek mskri...@redhat.com | To: Peter V. Saveliev p...@redhat.com, vdsm-devel@lists.fedorahosted.org Development | vdsm-devel@lists.fedorahosted.org | Sent: Monday, July 8, 2013 5:13:42 PM | Subject: Re: [vdsm] migration progress feature | | | On Jul 4, 2013, at 16:04 , Peter V. Saveliev p...@redhat.com wrote: | | … | | Goal | | | We have to implement a feature, migration progress bar in the UI. This | migration bar should reflect not only the progress, but if the migration | is stalled and so on. | | Tasks | = | | * Get the information from libvirt: it provides job progress in the same | way for all migration-like jobs: migration, suspend, snapshot | * Feed this information to the engine | * Reflect it in the UI | | API status | == | | Libvirt info is OK — it is available for any migration-like job, be it | migration, suspend or snapshot. | | In VDSM, we have an API call, a separate verb to report the migration | progress: migrateStatus() | | But also we have getVmList() call, polled by the engine every several | seconds. | | Proposal | | | We would propose to provide an optional field, `migrateStatus`, in the | report sent by getVmList(). This approach should save a good amount of | traffic and ease the engine side logic. | | Having the separate verb, this can sound weird, but I'm sure that the | optimisation worth it. | | Please, read the patchset [1] and discuss it here. | | From my point of view this makes sense as it doesn't require additional | bandwidth and calls. | Also means no additional complexity in engine. I don't feel comfortable to | add another poll request during migration which makes the logic in engine | more complicated and means even higher traffic between hosts and engine. Not | that it's not doable, I just don't think it's the right way to go. I agree | it's not ideal from purely API point of view...however to me it seems | logical to have it as part of statistics as it is a sort of statistic, and | it doesn't require an extra call. The extra call already exist, it's just | that I think we already have too many. We should start looking into | effective communication, not necessarily clean | | Thanks, | michal | +1 such a feature will be very handy when migrating large guests (8GB+) over sub-optimal network. In such a case the last thing you want to do is additional calls. Re-using the existing calls by extending the statistics makes a lot of sense. | | | Thanks. | | [1] http://gerrit.ovirt.org/#/c/15125/ | -- | Peter V. Saveliev | ___ | vdsm-devel mailing list | vdsm-devel@lists.fedorahosted.org | https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel | | ___ | vdsm-devel mailing list | vdsm-devel@lists.fedorahosted.org | https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel | ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
On 07/18/2013 01:24 AM, Doron Fediuck wrote: - Original Message - | From: Michal Skrivanek mskri...@redhat.com | To: Peter V. Saveliev p...@redhat.com, vdsm-devel@lists.fedorahosted.org Development | vdsm-devel@lists.fedorahosted.org | Sent: Monday, July 8, 2013 5:13:42 PM | Subject: Re: [vdsm] migration progress feature | | | On Jul 4, 2013, at 16:04 , Peter V. Saveliev p...@redhat.com wrote: | | … | | Goal | | | We have to implement a feature, migration progress bar in the UI. This | migration bar should reflect not only the progress, but if the migration | is stalled and so on. | | Tasks | = | | * Get the information from libvirt: it provides job progress in the same | way for all migration-like jobs: migration, suspend, snapshot | * Feed this information to the engine | * Reflect it in the UI | | API status | == | | Libvirt info is OK — it is available for any migration-like job, be it | migration, suspend or snapshot. | | In VDSM, we have an API call, a separate verb to report the migration | progress: migrateStatus() | | But also we have getVmList() call, polled by the engine every several | seconds. | | Proposal | | | We would propose to provide an optional field, `migrateStatus`, in the | report sent by getVmList(). This approach should save a good amount of | traffic and ease the engine side logic. | | Having the separate verb, this can sound weird, but I'm sure that the | optimisation worth it. | | Please, read the patchset [1] and discuss it here. | | From my point of view this makes sense as it doesn't require additional | bandwidth and calls. | Also means no additional complexity in engine. I don't feel comfortable to | add another poll request during migration which makes the logic in engine | more complicated and means even higher traffic between hosts and engine. Not | that it's not doable, I just don't think it's the right way to go. I agree | it's not ideal from purely API point of view...however to me it seems | logical to have it as part of statistics as it is a sort of statistic, and | it doesn't require an extra call. The extra call already exist, it's just | that I think we already have too many. We should start looking into | effective communication, not necessarily clean | | Thanks, | michal | +1 such a feature will be very handy when migrating large guests (8GB+) over sub-optimal network. In such a case the last thing you want to do is additional calls. Re-using the existing calls by extending the statistics makes a lot of sense. i'm also in favor - vm status is treated as an async update based on the stats. vdsm can send this item only for vms which are migrating for example. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
