Mike, From a CloudStack perspective, it will keep implementation specific concepts from the base data model, and provide a great test case to develop a mechanism to capture this information in 4.3. Ideally, I want CloudStack to exploit these implementation specific features. I just want to provide a facility to manage that data that does not leak across implementations.
Thanks, -John On Jun 10, 2013, at 6:07 PM, Mike Tutkowski <mike.tutkow...@solidfire.com> wrote: > I am pretty sure it is not a blocker for me to drop Burst IOPS. Let me run > that past Product Management, but I don't think it's a problem. > > > On Mon, Jun 10, 2013 at 4:01 PM, John Burwell <jburw...@basho.com> wrote: > >> Mike, >> >> Yes. I realize my other reply did not explicitly state leaving the Min >> and Max IOPS fields in the data model as these seem to generic terms across >> storage implementations. >> >> Thanks, >> -John >> >> On Jun 10, 2013, at 5:57 PM, Mike Tutkowski <mike.tutkow...@solidfire.com> >> wrote: >> >>> More generally speaking, you're looking to remove Burst IOPS from >>> CloudStack for 4.2, but we would keep Min and Max (and they would be >>> displayed in the Disk Offering dialog as I've proposed)? >>> >>> >>> On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski < >>> mike.tutkow...@solidfire.com> wrote: >>> >>>> Just to make sure I understand your request, are you looking to display >>>> Min and Max (as long as Wei's feature is not in use), but not display >> Burst >>>> IOPS? >>>> >>>> >>>> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jburw...@basho.com> >> wrote: >>>> >>>>> Mike, >>>>> >>>>> My concern becomes that we start ballooning the data model and user >>>>> interface with a fields that are documented as, "If using SolidFire, >> then >>>>> Burst IOPS is honored and foo and bar are not. For solution X, Burst >> IOPS >>>>> is ignored, but foo and bar apply." It may have to hold until 4.3, >> but it >>>>> seems like we need an extended data concept for storage drivers that >> allow >>>>> them to define an additional set of properties, and persist them into >> the >>>>> database as a JSON document. Such an enhancement would also require >> some >>>>> UI fanciness to consume the metadata provided by the driver and adjust >> the >>>>> UI. Would it be possible to defer Burst IOPS until 4.3 when we could >>>>> address extended driver data in a holistic manner? >>>>> >>>>> Thanks, >>>>> -John >>>>> >>>>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski < >> mike.tutkow...@solidfire.com> >>>>> wrote: >>>>> >>>>>> My thinking is that Min and Max are industry standard and Burst is a >> new >>>>>> concept that could catch on. >>>>>> >>>>>> >>>>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jburw...@basho.com> >>>>> wrote: >>>>>> >>>>>>> Wei, >>>>>>> >>>>>>> In this case, we can have the hypervisor or storage providing the >>>>> quality >>>>>>> of service guarantee. Naively, it seems reasonable to separate >>>>> hypervisor >>>>>>> and storage QoS parameters and enforce mutually exclusion. Not only >>>>> does >>>>>>> this approach simplify a whole range of operations, it also avoids >>>>>>> reconciling the data models. The only question I have about the >>>>> storage >>>>>>> provision IOPS is whether or not "Burst IOPS" is an industry standard >>>>> or >>>>>>> vendor specific concept. >>>>>>> >>>>>>> Thanks, >>>>>>> -John >>>>>>> >>>>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote: >>>>>>> >>>>>>>> Mike, >>>>>>>> I do not think users can select only one of them, as they are >>>>> implemented >>>>>>>> on different sides. >>>>>>>> Have you investigated the parameters other storage devices support, >>>>>>> besides >>>>>>>> min/max/burst IOPS? You'd better add all possible fields in your >>>>>>>> implementation. >>>>>>>> >>>>>>>> What do you think about this? >>>>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which >> includes >>>>> all >>>>>>>> supported storage vendors. >>>>>>>> If users select "SolidFire", min/max/burst IOPS will appear. >>>>>>>> If users select other vendors, relevant fields will appear. >>>>>>>> Actually I still insist that it is better to add the storage-related >>>>>>> fields >>>>>>>> in another table. >>>>>>>> >>>>>>>> -Wei >>>>>>>> >>>>>>>> >>>>>>>> 2013/6/10 Mike Tutkowski <mike.tutkow...@solidfire.com> >>>>>>>> >>>>>>>>> Here is my thinking: >>>>>>>>> >>>>>>>>> Two radio buttons (whatever we want to call them): >>>>>>>>> >>>>>>>>> 1) Hypervisor IOPS >>>>>>>>> 2) Storage IOPS >>>>>>>>> >>>>>>>>> Leave them both un-checked by default. >>>>>>>>> >>>>>>>>> If the user checks one or the other, the relevant fields appear. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski < >>>>>>>>> mike.tutkow...@solidfire.com> wrote: >>>>>>>>> >>>>>>>>>> What do you think, Wei? >>>>>>>>>> >>>>>>>>>> Should we come up with a way for only one feature (yours or mine) >>>>> to be >>>>>>>>>> used at a time on the new Disk Offering dialog? >>>>>>>>>> >>>>>>>>>> Since most storage-side provisioned IOPS don't break it down into >>>>>>>>> separate >>>>>>>>>> read and write categories, I think that's the way to go (only one >>>>>>> feature >>>>>>>>>> or the other). >>>>>>>>>> >>>>>>>>>> Any suggestions from a usability standpoint how we want to >> implement >>>>>>>>> this? >>>>>>>>>> It could be as simple as a radio button to turn on your feature >> and >>>>>>> mine >>>>>>>>>> off or vice versa. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jburw...@basho.com >>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Mike, >>>>>>>>>>> >>>>>>>>>>> I agree -- I can't image a situation where you would want to use >>>>> IOPS >>>>>>>>>>> provisioned by both the hypervisor and storage. There are two >>>>> points >>>>>>> of >>>>>>>>>>> concern -- the UI and the management server. We have to ensure >>>>> that >>>>>>> the >>>>>>>>>>> user can't create a VM from a compute/disk offering combination >>>>> where >>>>>>>>>>> hypervisor throttled I/O would contradict/conflict with storage >>>>>>>>> provisioned >>>>>>>>>>> IOPS. I think this functional conflict must be resolved in the >>>>>>>>> management >>>>>>>>>>> server to ensure that API calls are properly validated with a UX >>>>> that >>>>>>>>>>> avoids user confusion. Have Wei and you worked out an approach >> to >>>>>>>>>>> resolving this conflict? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> -John >>>>>>>>>>> >>>>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski < >>>>>>>>> mike.tutkow...@solidfire.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Wei has sent me the screen shots. >>>>>>>>>>>> >>>>>>>>>>>> I don't support Compute Offerings for 4.2, so that's not an >> issue >>>>>>>>> here. >>>>>>>>>>>> >>>>>>>>>>>> I do support Disk Offerings. >>>>>>>>>>>> >>>>>>>>>>>> It looks like Wei has added four new fields to the Disk >> Offering. >>>>>>>>>>>> >>>>>>>>>>>> I have added three (Min, Max, and Burst IOPS). >>>>>>>>>>>> >>>>>>>>>>>> We just need to decide if we should toggle between his and mine. >>>>>>>>>>>> >>>>>>>>>>>> I doubt a user would want to use both features at the same time. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell < >>>>> jburw...@basho.com> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Mike, >>>>>>>>>>>>> >>>>>>>>>>>>> Have Wei and you figured out the system level as well (e.g. >>>>> allowing >>>>>>>>>>>>> either storage provisioned IOPS or hypervisor throttling, but >> no >>>>>>>>> both)? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> -John >>>>>>>>>>>>> >>>>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski < >>>>>>>>>>> mike.tutkow...@solidfire.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Perhaps Wei could send me some screen shots of what he's >>>>> changed in >>>>>>>>>>> the >>>>>>>>>>>>> GUI >>>>>>>>>>>>>> for his feature? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM, John Burwell < >>>>> jburw...@basho.com >>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Wei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Have Mike Tutkowski and you reconciled the potential conflict >>>>>>>>>>> between a >>>>>>>>>>>>>>> throttled I/O VM and a provisioned IOPs volume? If so, what >>>>>>>>> solution >>>>>>>>>>>>> did >>>>>>>>>>>>>>> you select? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM, Wei ZHOU <ustcweiz...@gmail.com >>> >>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Guys, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into master. >>>>>>>>>>>>>>>> Please review the code on >> https://reviews.apache.org/r/11782 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If nobody object, I will merge into master in 72 hours. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -Wei >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <ustcweiz...@gmail.com> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling branch into >> master. >>>>>>>>>>>>>>>>> If nobody object, I will merge into master in 48 hours. >>>>>>>>>>>>>>>>> The purpose is : >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Virtual machines are running on the same storage device >>>>> (local >>>>>>>>>>> storage >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>> share strage). Because of the rate limitation of device >>>>> (such as >>>>>>>>>>>>> iops), >>>>>>>>>>>>>>> if >>>>>>>>>>>>>>>>> one VM has large disk operation, it may affect the disk >>>>>>>>>>> performance of >>>>>>>>>>>>>>>>> other VMs running on the same storage device. >>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate and limit the disk >>>>> I/O >>>>>>> of >>>>>>>>>>> VMs. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The feature includes: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in disk_offering, and >> global >>>>>>>>>>>>>>>>> configuration) >>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs >>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and iops) >>>>>>>>>>>>>>>>> JIRA ticket: >>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192 >>>>>>>>>>>>>>>>> FS (I will update later) : >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling >>>>>>>>>>>>>>>>> Merge check list :- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution success? >>>>>>>>>>>>>>>>> Yes >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> * Are there new dependencies introduced? >>>>>>>>>>>>>>>>> No >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> * What automated testing (unit and integration) is included >>>>> in >>>>>>>>> the >>>>>>>>>>> new >>>>>>>>>>>>>>>>> feature? >>>>>>>>>>>>>>>>> Unit tests are added. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> * What testing has been done to check for potential >>>>> regressions? >>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI. >>>>>>>>>>>>>>>>> (2) VM operations, including >>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, >>>>> restore >>>>>>>>>>>>>>>>> (3) Volume operations, including >>>>>>>>>>>>>>>>> Attach, Detach >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> To review the code, you can try >>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7 >>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>> Wei >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling >>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling >>>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301< >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071 >>>>>>>>>>>> (CLOUDSTACK-1301 >>>>>>>>>>>>> - >>>>>>>>>>>>>>> VM Disk I/O Throttling) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>> e: mike.tutkow...@solidfire.com >>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>>>>>>>>>> *™* >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>> e: mike.tutkow...@solidfire.com >>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>>>>>>>> *™* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Mike Tutkowski* >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>> e: mike.tutkow...@solidfire.com >>>>>>>>>> o: 303.746.7302 >>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>> http://solidfire.com/solution/overview/?video=play> >>>>>>>>>> *™* >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Mike Tutkowski* >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>> e: mike.tutkow...@solidfire.com >>>>>>>>> o: 303.746.7302 >>>>>>>>> Advancing the way the world uses the >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>>>>> *™* >>>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Tutkowski* >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>> e: mike.tutkow...@solidfire.com >>>>>> o: 303.746.7302 >>>>>> Advancing the way the world uses the >>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>> *™* >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Mike Tutkowski* >>>> *Senior CloudStack Developer, SolidFire Inc.* >>>> e: mike.tutkow...@solidfire.com >>>> o: 303.746.7302 >>>> Advancing the way the world uses the cloud< >> http://solidfire.com/solution/overview/?video=play> >>>> *™* >>>> >>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkow...@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the >>> cloud<http://solidfire.com/solution/overview/?video=play> >>> *™* >> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *™*
