Hi folks, There seems to be no objection to add four objects proposed by lan, and I believe my understanding about the latency is correct.
So we’ll upload the updated document soon to move forward. If you have any other comments, please let us know as soon as possible. Thank you. Hirochika On Sep 30, 2014, at 9:06 PM, Hirochika Asai <[email protected]> wrote: > Hi, > > Thank you for your contribution, and sorry for my delayed response. > > I think they are useful information for hypervisor operators, > so I personally support to add these objects to the MIB definition. > > However, I would like to clarify two objects related to access latency, > vmStorageReadLatency and vmStorageWriteLatency, from the > viewpoint of the implementation. > > If I understand the common behavior of VM’s disk access correctly, > it consists of the following steps. > > 1. A VM sets up a virtual AHCI command structure for disk access. > 2. The VM issues hypervisor call (and the VM suspends). > 3. The hypervisor schedules a virtual disk handling process/thread or > something equivalent. > 4. The process/thread processes the queued request and does something to > physical storage system. > 5. The process/thread receives a reply from the physical storage system. > 6. The process/thread constructs a virtual AHCI reply message structure for > the VM. > 7. The hypervisor schedules the VM to be resumed. > > According to my understanding of lan’s suggestion about latency is to measure > the > time from Step 4 to 6, and I think it could be implemented (although not yet > implemented > in Xen or KVM as far as I know). However, if you mean to measure the latency > from > Step 1 to 7, I don’t think it measurable from hypervisors. > > Is my understanding correct? > > Thank you. > Hirochika > > > On Sep 29, 2014, at 5:07 PM, Juergen Schoenwaelder > <[email protected]> wrote: > >> Hi, >> >> for me, it is most important to finish up. This project is ongoing for >> quite some time meanwhile and I think it is time to deliver. MIB >> moduels can be extended after publication as Proposed Standard. I am >> OK to accept these modifications if we at the same time close the door >> for new features and instead wrap up this MIB module. >> >> /js >> >> On Mon, Sep 29, 2014 at 04:27:13PM +0900, Keiichi SHIMA wrote: >>> Hello Ian, >>> >>> Thank you for contributing the text. I personally think the text is fine. >>> Let's wait for a while if we receive any comments about the text from other >>> authors and WG people. >>> >>> I believe this modification doesn't affect the current consensus we have >>> about the draft. If the text is accepted by the WG, then we'll integrate >>> it to the draft and would like to move forward. >>> >>> Regards, >>> --- >>> Keiichi SHIMA (島 慶一) >>> WIDE project <[email protected]> >>> Research Laboratory, IIJ Innovation Institute, Inc <[email protected]> >>> >>> >>> >>> On 2014/09/29, at 15:33, Ian West <[email protected]> wrote: >>> >>>> Attached is a diff against the text form of the draft, I believe I have >>>> addressed the changes that >>>> might be required. >>>> >>>> Also for convenience the actual text of the proposed additional counters >>>> is as follows. >>>> >>>> vmStorageReadOctets OBJECT-TYPE >>>> SYNTAX Counter64 >>>> MAX-ACCESS read-only >>>> STATUS current >>>> DESCRIPTION >>>> "The total number of bytes read from this device. >>>> Discontinuities in the value of this counter can occur >>>> at re-initialization of the hypervisor, and >>>> administrative state (vmAdminState) changes of the >>>> virtual machine." >>>> ::= { vmStorageEntry 15 } >>>> >>>> vmStorageWriteOctets OBJECT-TYPE >>>> SYNTAX Counter64 >>>> MAX-ACCESS read-only >>>> STATUS current >>>> DESCRIPTION >>>> "The total number of bytes written to this device. >>>> Discontinuities in the value of this counter can occur >>>> at re-initialization of the hypervisor, and >>>> administrative state (vmAdminState) changes of the >>>> virtual machine." >>>> ::= { vmStorageEntry 16 } >>>> >>>> vmStorageReadLatency OBJECT-TYPE >>>> SYNTAX Counter64 >>>> MAX-ACCESS read-only >>>> STATUS current >>>> DESCRIPTION >>>> "The total number of microseconds read requests have been >>>> queued for this device. >>>> This would typically be implemented by storing the high >>>> precision >>>> system time stamp of when the request is received from the >>>> Virtual >>>> Machine with the request, the difference between this initial >>>> time >>>> stamp and the time at which the requested operation has >>>> completed >>>> should be converted to microseconds and accumulated. >>>> Discontinuities in the value of this counter can occur >>>> at re-initialization of the hypervisor, and >>>> administrative state (vmAdminState) changes of the >>>> virtual machine." >>>> ::= { vmStorageEntry 17 } >>>> >>>> vmStorageWriteLatency OBJECT-TYPE >>>> SYNTAX Counter64 >>>> MAX-ACCESS read-only >>>> STATUS current >>>> DESCRIPTION >>>> "The total number of microseconds write requests have been >>>> queued for this device. >>>> This would typically be implemented by storing the high >>>> precision >>>> system time stamp of when the request is received from the >>>> Virtual >>>> Machine with the request, the difference between this initial >>>> time >>>> stamp and the time at which the requested operation has >>>> completed >>>> should be converted to microseconds and accumulated. >>>> Discontinuities in the value of this counter can occur >>>> at re-initialization of the hypervisor, and >>>> administrative state (vmAdminState) changes of the >>>> virtual machine." >>>> ::= { vmStorageEntry 18 } >>>> >>>> Thankyou again for your consideration and feedback. >>>> >>>> Regards, >>>> Ian West. >>>> >>>> -----Original Message----- >>>> From: Juergen Schoenwaelder [mailto:[email protected]] >>>> Sent: Monday, 22 September 2014 15:25 >>>> To: Keiichi SHIMA >>>> Cc: Ian West; [email protected]; ASAI Hirochika; SEKIYA Yuji; Tina Tsou; >>>> Cathy Zhou; ESAKI Hiroshi; Michael MacFaden >>>> Subject: Re: [OPSAWG] Comments from niw.com.au operator on draft on >>>> >>>> Hi, >>>> >>>> it would help if people would propose concrete text of what the >>>> description of the counters would be. What is most important is that >>>> we define counters that are implementable in a meaningful way on a >>>> large set of different hypervisors. >>>> >>>> /js >>>> >>>> On Sun, Sep 21, 2014 at 09:36:55PM +0900, Keiichi SHIMA wrote: >>>>> Hello Ian, >>>>> >>>>>> Your description of the accumulated milliseconds (or potentially >>>>>> microseconds to allow for things getting a lot faster) is exactly >>>>>> what I had in mind. The delta of this figure divided by the delta iops, >>>>>> should give an approximate measure of the load the io >>>>>> subsystem is under. >>>>> >>>>> Thank you for the clarification. I got better understanding of your >>>>> proposal. I tend to agree that the above usage may be useful. >>>>> >>>>> I again don’t have any strong support/objection for adding the above new >>>>> entry to the VMM-MIB. I’d like to hear if other authors and people in >>>>> this WG support this. >>>>> >>>>> Regards, >>>>> --- >>>>> Keiichi SHIMA (島 慶一) >>>>> WIDE project <[email protected]> >>>>> Research Laboratory, IIJ Innovation Institute, Inc <[email protected]> >>>>> >>>>> >>>>> >>>>> 2014/09/21 11:27、Ian West <[email protected]> のメール： >>>>> >>>>>> Hello, >>>>>> >>>>>> Thankyou for your clarification and feedback. >>>>>> >>>>>> Your description of the accumulated milliseconds (or potentially >>>>>> microseconds to allow for things getting a lot faster) is exactly >>>>>> what I had in mind. The delta of this figure divided by the delta iops, >>>>>> should give an approximate measure of the load the io >>>>>> subsystem is under. >>>>>> >>>>>> Regards, >>>>>> Ian West. >>>>>> >>>>>> -----Original Message----- >>>>>> From: Keiichi SHIMA [mailto:[email protected]] >>>>>> Sent: Saturday, 20 September 2014 18:03 >>>>>> To: [email protected] >>>>>> Cc: SHIMA Keiichi; ASAI Hirochika; Jürgen Schönwälder; SEKIYA Yuji; Tina >>>>>> Tsou; Cathy Zhou; ESAKI Hiroshi; Ian West; Michael MacFaden >>>>>> Subject: Re: [OPSAWG] Comments from niw.com.au operator on draft on >>>>>> >>>>>> Hello, >>>>>> >>>>>>> It would be really nice to have IO service times as well, ie something >>>>>>> like accumulated milliseconds of wait time for io servicing, >>>>>>> this might be a harder to instrument. A counter64 of cumulative ms >>>>>>> (us?) awaiting delivery is a really handy read/write latency metric. >>>>>> >>>>>> Let me clarify the above sentence. Is the service time, you are talking >>>>>> about, the time between an I/O request arrives to the storage subsystem >>>>>> (in a hypervisor) and the time its response is returned to the calling >>>>>> VM? And you want to add an entry to keep an accumulated value of those >>>>>> I/O request delay? >>>>>> >>>>>> >>>>>>> The vmStorageEntry statistics while having read operation and write >>>>>>> operation counters don't allow for measuring the volume of read and >>>>>>> write traffic. It would be useful to include 64 bit counters for read >>>>>>> and write octets. >>>>>> >>>>>> I don’t have any particular opinion on this topic. If people in this WG >>>>>> support adding the above entries, then maybe it is possible to add them. >>>>>> >>>>>> Regards, >>>>>> --- >>>>>> Keiichi SHIMA (島 慶一) >>>>>> Research Laboratory, IIJ Innovation Institute, Inc <[email protected]> >>>>>> WIDE project <[email protected]> >>>>>> >>>>>> >>>>>> >>>>>> 2014/09/18 2:45、Michael MacFaden <[email protected]> のメール： >>>>>> >>>>>>> Hi authors, >>>>>>> >>>>>>> Regarding: https://datatracker.ietf.org/doc/draft-ietf-opsawg-vmm-mib/ >>>>>>> >>>>>>> Ian as allowed me to forward you his comments on the draft. >>>>>>> >>>>>>> Cheers, >>>>>>> Mike MacFaden >>>>>>> >>>>>>> >>>>>>> From: Ian West <[email protected]> >>>>>>> Subject: RE: Mail regarding draft-ietf-opsawg-vmm-mib >>>>>>> Date: September 16, 2014 at 3:18:08 AM PDT >>>>>>> To: 'Michael MacFaden' <[email protected]> >>>>>>> >>>>>>> >>>>>>> Overall I think it looks great. >>>>>>> >>>>>>> It would be really nice to have IO service times as well, ie something >>>>>>> like accumulated milliseconds of wait time for io servicing, >>>>>>> this might be a harder to instrument. A counter64 of cumulative ms >>>>>>> (us?) awaiting delivery is a really handy read/write latency metric. >>>>>>> >>>>>>> The cputime and memory figures look good. >>>>>>> >>>>>>> I am thinking there would be a storage object for the swap space which >>>>>>> would give most of the paging stats if actual byte counts were added ? >>>>>>> >>>>>>> Links back to the ifindex should also tie into vswitch I think ? My >>>>>>> current walks of vmware are nesting vswitch ports nicely so that’s >>>>>>> looking good. Network stats seem good already, its primarily the per >>>>>>> vhost cpu usage and io stats that are weak from a real time >>>>>>> perspective, this MIB looks like it should cover it really well. >>>>>>> >>>>>>> Is this intended to be a vsphere (central management) resource, or a >>>>>>> per physical host (or both, which would be great) ? >>>>>>> >>>>>>> If you were to query this from a vsphere central management host, I am >>>>>>> thinking you might want to add to the vmtable an identifier >>>>>>> for the physical host that the vm is currently running on. This would >>>>>>> make the table useful either standalone or across a bunch of >>>>>>> physical hosts. >>>>>>> >>>>>>> Ill think on it some more, this seems like a great step forward over >>>>>>> whats there now already in VMware’s implementation. >>>>>>> >>>>>>> >>>>>>> The vmStorageEntry statistics while having read operation and write >>>>>>> operation counters don't allow for measuring the volume of read and >>>>>>> write traffic. It would be useful to include 64 bit counters for read >>>>>>> and write octets. >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> OPSAWG mailing list >>>>>>> [email protected] >>>>>>> https://www.ietf.org/mailman/listinfo/opsawg >>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> Juergen Schoenwaelder Jacobs University Bremen gGmbH >>>> Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany >>>> Fax: +49 421 200 3103 <http://www.jacobs-university.de/> >>>> >>>> <draft-ietf-opsawg-vmm-mib-01.diff> >>> >> >> -- >> Juergen Schoenwaelder Jacobs University Bremen gGmbH >> Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany >> Fax: +49 421 200 3103 <http://www.jacobs-university.de/> > > -- > Hirochika Asai <[email protected]>, The University of Tokyo > > _______________________________________________ > OPSAWG mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/opsawg -- Hirochika Asai <[email protected]>, The University of Tokyo _______________________________________________ OPSAWG mailing list [email protected] https://www.ietf.org/mailman/listinfo/opsawg
