Yunsong (Roamer) Lu wrote: > deepti dhokte - Sun Microsystems - Menlo Park United States wrote: > >> I recall, some conversations whether rings belonging to same >> group have same interrupt number so that blanking that shared >> interrupt, can emulate polling of a group. >> My original question also, said that can you mask it in a framework, >> such as even if individual rings generate different interrupts, it can >> be delivered as some common virtual interrupt to upper layer, >> so polling on ring-group could work. >> cause, if drive rimplementation doesn't support it, can we implement >> it in mac framework? > > I don't understand why you want to do this.
explained below. > If you look at the document, you may see that the per-group interrupt > is nice-to-have feature *ONLY* when there are not enough per-ring > interrupts. my point is per-group single interrupt may not be just "nice-to-have" and it has potential and use-cases that should evolve it into "important" feature. > What's your case that need to poll a ring group even though there are > per-ring interrupts implemented? > The particular case for which I see, per-group interrupt useful is that - ring-groups are used for load-balancing in heavy network conditions. so traffic on all the rings of a group is buffering same packet-flow. so to make interrupt blanking work so that this heavy network traffic don't keep thrashing interrupt on the bound cpus, it would be good to have single interrupt across all rings of the same group. Otherwise you will have to blank individual interrupts of each ring of the group, which seems to me overhead. if single interrupts across multiple rings of the same group is supported in hardware and abstract from L2 framework then no virtual masking would be needed, and would be cleaner and simpler. but if there is no such support I was wondering if there can be something like virtual masking, i.e. either 1) the way I had earlier pictured and explained in earlier email's first paragraph. or 2) in which although interrupt 1,2,3,4 belong to rings of same group blanking 4, always would blank 1,2,3 as well. unless you guys have some other ways in mind. >> In the absence of multiple RX/TX rings support in hardware, >> does crossbow emulate RX/TX multiple rings/groups behavior? >> or are there plans to extend it in future phases? >> can SRS (softt ring Set) help us to achieve that in future? > > Why the framework need to pretend having multiple hardware ring and/or > ring group? What's the behavior the framework need to emulate? why network interface card care to have multiple rings or groups? that's the same reason why in absence of such a feature in hardware why we would want to emulate it in software. afaik, current crossbow, if there are no multiple RX rings/channels/.fifo to program the rule/policy on it , have soft rings, that's the same reason you would want to emulate such a behavior (ring-groups) in mac framework. and wondered if SRS does it? > >> aren't they just slightly different in just one structure member >> i.e. mr_poll and mr_send? > > Firstly, to combine these two data structures is unnatural for now. > Two function prototypes are different, so I don't know what's the > necessity to let them share one functional pointer. I leave it upto you. but I still think, you may want to remove redundany in code. structure members of these 2 structures, are same except for the last member. unless I am missing something. If I am, please correct me. > >>> The ring grouping isolate VNICs from hardware layer, and the >>> virtualization is done at layer 2 instead of L3/L4. >> >> >> Having user-prgrammed Rx rings(/flows) with L3/L4 classifications >> on top of VNIC/NIC, would differentiate crossbow features over >> other vendor's crossbow-like virtualization capabilities. >> >> It would be especially useful, when you want to run multiple >> applications such >> as streaming server(RTP), DHCP server (UDP)on same Virtual machine >> with a VNIC. >> such as no single app can monopolize given VNIC's already capped >> bandwidth. >> >> rx-ring grouping gives many benefits and I am not sure if we want to >> limit >> it to just VNIC/L2 classification. >> >> so I am trying to think-out loud here to wonder it's further >> exploitation >> in line with VNICs. >> >>> Ring group will be the basic entity of virtualizing hardware resources. >>> >>> >> programming ring-group for L3/L4 classifications can help >> prformance of network virtual machines, I suppose. > > Looks the document is not clear enough on explaining this. We'll > update it accordingly. ok. > > Briefly, the device virtualization is done at layer 2, through MAC > addresses and VLAN, and we'll create VNIC on top of this type of > vitualization. L3/L4 hardware classifications are still important, but > that will happen only within a receive ring group, and FLOW is created > with L3/L4 classification rules. We're not dropping any helpful > feautre, but organizing the sources in a natural way. ok. > >>>> 5) >>>> Can you group rings of different physical NICs, If yes, what is the >>>> interface for the same? >>> >>> No. You may check out the ring group definition in the document. :) >>> >> so it's hardware design limitation of driver implementation limitation.? >> btw, I don't see the definition, can you help me with a page number? > > Page 3. Thanks. > > In this design, the ring group is mapped to hardware resources, so > it's impossible to group rings from different devices. I don't know if its' impossible but as network interfaces/devices and their driver implementations evolve, there might be one in the making. wishful thinking ;-) > > But as mentioned by Kais, when creating VNIC on top of aggr, the aggr > driver may re-group those hardware groups for its client, say VNIC. > From the VNIC's point of view, the ring group exposed by aggr driver > may include rings from different hardware instances. yeah, he explained that use-case in his email. > > We need to further discuss this as Kais suggested. ok, -Deepti > > Thanks, > > Roamer
