Re: [vdsm] migration progress feature
On Jul 22, 2013, at 15:03 , Dan Kenigsberg wrote: > On Sun, Jul 21, 2013 at 10:29:03PM +0300, Itamar Heim wrote: >> On 07/21/2013 12:14 PM, Dan Kenigsberg wrote: >>> Introducing a new verb, getMigrationStatuses(), would allow Engine to >>> collect this piece of statistics in the exact frequency that it needs >>> it. Coupling this data with the frequently-polled state of VM begs for >>> future complaints that getVMList(full=False) is too heavy, and that we >>> must introduce getVMList(full=False, veryveryskinny=True). >> >> so the most likely outcome is engine would be calling this every >> time it calls getAllVmStats? > > Having two different verbs allows the client to play with the frequency > of polling. Last word I've heard was that getAllVmStats is too heavy to > be called in the frequency that getMigrationStatuses is needed. > > If Engine is happy to lock the frequency of getMigrationStatuses to that > of getAllVmStats, we can include migrateStatus within getAllVmStats. I > already conceded to this earlier in this thread. which is possible, yes. The complaint is that it still wastes a roundtrip. And user-wise it would be too slow, I as a user would expect the percentage move a bit faster than once in 15s. The frequency of "list" is actually ideal (3s). We really do have to much of polling and we should not add more. Still, let's see after Vinzenz provides some example measurements of the real traffic/load it causes. I hope it will give us a hint which way is better. Before that I'd hold off with implementation as I don't think it's wise to implement something which may cause more performance issues than we already have. Thanks, michal > > Dan. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
On Sun, Jul 21, 2013 at 10:29:03PM +0300, Itamar Heim wrote: > On 07/21/2013 12:14 PM, Dan Kenigsberg wrote: > >Introducing a new verb, getMigrationStatuses(), would allow Engine to > >collect this piece of statistics in the exact frequency that it needs > >it. Coupling this data with the frequently-polled state of VM begs for > >future complaints that getVMList(full=False) is too heavy, and that we > >must introduce getVMList(full=False, veryveryskinny=True). > > so the most likely outcome is engine would be calling this every > time it calls getAllVmStats? Having two different verbs allows the client to play with the frequency of polling. Last word I've heard was that getAllVmStats is too heavy to be called in the frequency that getMigrationStatuses is needed. If Engine is happy to lock the frequency of getMigrationStatuses to that of getAllVmStats, we can include migrateStatus within getAllVmStats. I already conceded to this earlier in this thread. Dan. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
On 07/21/2013 12:14 PM, Dan Kenigsberg wrote: Introducing a new verb, getMigrationStatuses(), would allow Engine to collect this piece of statistics in the exact frequency that it needs it. Coupling this data with the frequently-polled state of VM begs for future complaints that getVMList(full=False) is too heavy, and that we must introduce getVMList(full=False, veryveryskinny=True). so the most likely outcome is engine would be calling this every time it calls getAllVmStats? ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
On Thu, Jul 18, 2013 at 05:29:14AM +0300, Itamar Heim wrote: > On 07/18/2013 01:24 AM, Doron Fediuck wrote: > > > > > >- Original Message - > >| From: "Michal Skrivanek" > >| To: "Peter V. Saveliev" , > >"vdsm-devel@lists.fedorahosted.org Development" > >| > >| Sent: Monday, July 8, 2013 5:13:42 PM > >| Subject: Re: [vdsm] migration progress feature > >| > >| > >| On Jul 4, 2013, at 16:04 , Peter V. Saveliev wrote: > >| > >| > … > >| > > >| > Goal > >| > > >| > > >| > We have to implement a feature, migration progress bar in the UI. This > >| > migration bar should reflect not only the progress, but if the migration > >| > is stalled and so on. > >| > > >| > Tasks > >| > = > >| > > >| > * Get the information from libvirt: it provides job progress in the same > >| > way for all migration-like jobs: migration, suspend, snapshot > >| > * Feed this information to the engine > >| > * Reflect it in the UI > >| > > >| > API status > >| > == > >| > > >| > Libvirt info is OK — it is available for any migration-like job, be it > >| > migration, suspend or snapshot. > >| > > >| > In VDSM, we have an API call, a separate verb to report the migration > >| > progress: migrateStatus() > >| > > >| > But also we have getVmList() call, polled by the engine every several > >| > seconds. > >| > > >| > Proposal > >| > > >| > > >| > We would propose to provide an optional field, `migrateStatus`, in the > >| > report sent by getVmList(). This approach should save a good amount of > >| > traffic and ease the engine side logic. > >| > > >| > Having the separate verb, this can sound weird, but I'm sure that the > >| > optimisation worth it. > >| > > >| > Please, read the patchset [1] and discuss it here. > >| > >| From my point of view this makes sense as it doesn't require additional > >| bandwidth and calls. > >| Also means no additional complexity in engine. I don't feel comfortable to > >| add another poll request during migration which makes the logic in engine > >| more complicated and means even higher traffic between hosts and engine. > >Not > >| that it's not doable, I just don't think it's the right way to go. I agree > >| it's not ideal from purely API point of view...however to me it seems > >| logical to have it as part of statistics as it is a sort of statistic, and > >| it doesn't require an extra call. The extra call already exist, it's just > >| that I think we already have too many. We should start looking into > >| effective communication, not necessarily clean > >| > >| Thanks, > >| michal > >| > > > >+1 > >such a feature will be very handy when migrating large guests (8GB+) over > >sub-optimal network. In such a case the last thing you want to do is > >additional > >calls. Re-using the existing calls by extending the statistics makes a lot > >of sense. > > i'm also in favor - vm status is treated as an async update based on > the stats. vdsm can send this item only for vms which are migrating > for example. I oppose the idea of overloading the getVMList(full=False) with migration-related statistics just because Engine happens to be calling this verb frequently. The full=False was born in sin and defies common practices of client/server design. getAllVmStats() was intended to supply all interesting statistics for all running VMs. Adding migration statistics to it makes much more sense. However, the size of its repsonse is quite large already, and I heard that people are afread that calling it more frequestly would degrade Engine's performence. Introducing a new verb, getMigrationStatuses(), would allow Engine to collect this piece of statistics in the exact frequency that it needs it. Coupling this data with the frequently-polled state of VM begs for future complaints that getVMList(full=False) is too heavy, and that we must introduce getVMList(full=False, veryveryskinny=True). Regrads, Dan. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
On 07/18/2013 01:24 AM, Doron Fediuck wrote: - Original Message - | From: "Michal Skrivanek" | To: "Peter V. Saveliev" , "vdsm-devel@lists.fedorahosted.org Development" | | Sent: Monday, July 8, 2013 5:13:42 PM | Subject: Re: [vdsm] migration progress feature | | | On Jul 4, 2013, at 16:04 , Peter V. Saveliev wrote: | | > … | > | > Goal | > | > | > We have to implement a feature, migration progress bar in the UI. This | > migration bar should reflect not only the progress, but if the migration | > is stalled and so on. | > | > Tasks | > = | > | > * Get the information from libvirt: it provides job progress in the same | > way for all migration-like jobs: migration, suspend, snapshot | > * Feed this information to the engine | > * Reflect it in the UI | > | > API status | > == | > | > Libvirt info is OK — it is available for any migration-like job, be it | > migration, suspend or snapshot. | > | > In VDSM, we have an API call, a separate verb to report the migration | > progress: migrateStatus() | > | > But also we have getVmList() call, polled by the engine every several | > seconds. | > | > Proposal | > | > | > We would propose to provide an optional field, `migrateStatus`, in the | > report sent by getVmList(). This approach should save a good amount of | > traffic and ease the engine side logic. | > | > Having the separate verb, this can sound weird, but I'm sure that the | > optimisation worth it. | > | > Please, read the patchset [1] and discuss it here. | | From my point of view this makes sense as it doesn't require additional | bandwidth and calls. | Also means no additional complexity in engine. I don't feel comfortable to | add another poll request during migration which makes the logic in engine | more complicated and means even higher traffic between hosts and engine. Not | that it's not doable, I just don't think it's the right way to go. I agree | it's not ideal from purely API point of view...however to me it seems | logical to have it as part of statistics as it is a sort of statistic, and | it doesn't require an extra call. The extra call already exist, it's just | that I think we already have too many. We should start looking into | effective communication, not necessarily clean | | Thanks, | michal | +1 such a feature will be very handy when migrating large guests (8GB+) over sub-optimal network. In such a case the last thing you want to do is additional calls. Re-using the existing calls by extending the statistics makes a lot of sense. i'm also in favor - vm status is treated as an async update based on the stats. vdsm can send this item only for vms which are migrating for example. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
- Original Message - | From: "Michal Skrivanek" | To: "Peter V. Saveliev" , "vdsm-devel@lists.fedorahosted.org Development" | | Sent: Monday, July 8, 2013 5:13:42 PM | Subject: Re: [vdsm] migration progress feature | | | On Jul 4, 2013, at 16:04 , Peter V. Saveliev wrote: | | > … | > | > Goal | > | > | > We have to implement a feature, migration progress bar in the UI. This | > migration bar should reflect not only the progress, but if the migration | > is stalled and so on. | > | > Tasks | > = | > | > * Get the information from libvirt: it provides job progress in the same | > way for all migration-like jobs: migration, suspend, snapshot | > * Feed this information to the engine | > * Reflect it in the UI | > | > API status | > == | > | > Libvirt info is OK — it is available for any migration-like job, be it | > migration, suspend or snapshot. | > | > In VDSM, we have an API call, a separate verb to report the migration | > progress: migrateStatus() | > | > But also we have getVmList() call, polled by the engine every several | > seconds. | > | > Proposal | > | > | > We would propose to provide an optional field, `migrateStatus`, in the | > report sent by getVmList(). This approach should save a good amount of | > traffic and ease the engine side logic. | > | > Having the separate verb, this can sound weird, but I'm sure that the | > optimisation worth it. | > | > Please, read the patchset [1] and discuss it here. | | From my point of view this makes sense as it doesn't require additional | bandwidth and calls. | Also means no additional complexity in engine. I don't feel comfortable to | add another poll request during migration which makes the logic in engine | more complicated and means even higher traffic between hosts and engine. Not | that it's not doable, I just don't think it's the right way to go. I agree | it's not ideal from purely API point of view...however to me it seems | logical to have it as part of statistics as it is a sort of statistic, and | it doesn't require an extra call. The extra call already exist, it's just | that I think we already have too many. We should start looking into | effective communication, not necessarily clean | | Thanks, | michal | +1 such a feature will be very handy when migrating large guests (8GB+) over sub-optimal network. In such a case the last thing you want to do is additional calls. Re-using the existing calls by extending the statistics makes a lot of sense. | > | > | > Thanks. | > | > [1] http://gerrit.ovirt.org/#/c/15125/ | > -- | > Peter V. Saveliev | > ___ | > vdsm-devel mailing list | > vdsm-devel@lists.fedorahosted.org | > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel | | ___ | vdsm-devel mailing list | vdsm-devel@lists.fedorahosted.org | https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel | ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] migration progress feature
On Jul 4, 2013, at 16:04 , Peter V. Saveliev wrote: > … > > Goal > > > We have to implement a feature, migration progress bar in the UI. This > migration bar should reflect not only the progress, but if the migration is > stalled and so on. > > Tasks > = > > * Get the information from libvirt: it provides job progress in the same way > for all migration-like jobs: migration, suspend, snapshot > * Feed this information to the engine > * Reflect it in the UI > > API status > == > > Libvirt info is OK — it is available for any migration-like job, be it > migration, suspend or snapshot. > > In VDSM, we have an API call, a separate verb to report the migration > progress: migrateStatus() > > But also we have getVmList() call, polled by the engine every several seconds. > > Proposal > > > We would propose to provide an optional field, `migrateStatus`, in the report > sent by getVmList(). This approach should save a good amount of traffic and > ease the engine side logic. > > Having the separate verb, this can sound weird, but I'm sure that the > optimisation worth it. > > Please, read the patchset [1] and discuss it here. From my point of view this makes sense as it doesn't require additional bandwidth and calls. Also means no additional complexity in engine. I don't feel comfortable to add another poll request during migration which makes the logic in engine more complicated and means even higher traffic between hosts and engine. Not that it's not doable, I just don't think it's the right way to go. I agree it's not ideal from purely API point of view...however to me it seems logical to have it as part of statistics as it is a sort of statistic, and it doesn't require an extra call. The extra call already exist, it's just that I think we already have too many. We should start looking into effective communication, not necessarily clean Thanks, michal > > > Thanks. > > [1] http://gerrit.ovirt.org/#/c/15125/ > -- > Peter V. Saveliev > ___ > vdsm-devel mailing list > vdsm-devel@lists.fedorahosted.org > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] migration progress feature
… Goal We have to implement a feature, migration progress bar in the UI. This migration bar should reflect not only the progress, but if the migration is stalled and so on. Tasks = * Get the information from libvirt: it provides job progress in the same way for all migration-like jobs: migration, suspend, snapshot * Feed this information to the engine * Reflect it in the UI API status == Libvirt info is OK — it is available for any migration-like job, be it migration, suspend or snapshot. In VDSM, we have an API call, a separate verb to report the migration progress: migrateStatus() But also we have getVmList() call, polled by the engine every several seconds. Proposal We would propose to provide an optional field, `migrateStatus`, in the report sent by getVmList(). This approach should save a good amount of traffic and ease the engine side logic. Having the separate verb, this can sound weird, but I'm sure that the optimisation worth it. Please, read the patchset [1] and discuss it here. Thanks. [1] http://gerrit.ovirt.org/#/c/15125/ -- Peter V. Saveliev ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
