Yunsong (Roamer) Lu wrote: >Thank you for the comments! > > > >>1) >>This documents describes polling single ring. >>Can you poll group of rings, if they all share common interrupt number? >> >> >Polling can only happen on single ring, regardless sharing interrupt or >not. Practically, if a shared group interrupt is used, to poll a ring >group is just to poll all single rings separately. > > I recall, some conversations whether rings belonging to same group have same interrupt number so that blanking that shared interrupt, can emulate polling of a group. My original question also, said that can you mask it in a framework, such as even if individual rings generate different interrupts, it can be delivered as some common virtual interrupt to upper layer, so polling on ring-group could work. cause, if drive rimplementation doesn't support it, can we implement it in mac framework?
> > >>mrg_intr is described to be "nice to have" per group based common >>interrupt number, Is it driver dependent? or the mac framework >>can have virtual interrupt that masks individual interrupts of each >>individual ring of given group? >> >> >Driver need to make the decision > so you mean it is NIC/hardware limited factor or driver implementation specific? > whether to enable per-group interrupt >if there are no enough interrupts for all rings, > so, the interrupts may be limited in number. In that case sharing same interrupt among multiple rings of the group should help, isn't it? > because this depends on >how the hardware and driver organize interrupt sources. > > > I guess, driver should have ability to program interrupt vector and assigning interrupts, I may be missing something? >>2) >>If any hardware/network driver does not have ring support, >>can crossbow for such drivers emulate channel/Fifo/ring behavior in >>software? >> >> >The term "Ring" is used to describe a software data structure that is >mapped to hardware traffic channel, receive or transmit, > I understand that. >so any NIC has >at least 1 receive "ring" and 1 transmit "ring". > > yeah. >>Does SRS would serve that purpose? >> >> >SRS is on top of hardware ring. > > > In the absence of multiple RX/TX rings support in hardware, does crossbow emulate RX/TX multiple rings/groups behavior? or are there plans to extend it in future phases? can SRS (softt ring Set) help us to achieve that in future? >>3) >>I see there is mac_rx_ring_info_t and mac_rx_ring_group_info_t. >>how about if you have common structures for rx and tx side for info? >>Instead of having mac_rx_ring_info_t and mac_rx_ring_group_info_t >>would it make sense to have mac_ring_info_t and mac_ring_group_info_t >>to be usable for rx and tx side rings or ring-groups.? >> >>e.g. To implement above you can have mac_cb function pointer >>in mac_ring_info_t , and say - >>1) for rx side "mac_cb" can be initialized as "mr_poll" and >>2) on Tx side "mac_cb" can be initialized as "mr_send" >>since mr_driver, mr_intr, mr_start, mr_stop are members of >>mac_rx_ring_info_t as well as mac_tx_ring_info_t and it's just that >>mr_poll and mr_send routines are different for rx and tx side ring_info >>respectively. >> >> >(*mr_poll)() and (*mr_send)() have different prototypes, and they could >evolve in the future. More functions are possibly to be added to either >data structure. > >The two different data structures, mac_rx_ring_info_t and >mac_tx_ring_info_t, are used to simplify the interface. > They have 4 out of 5 redundant members. > But if we can >make sure the two data structures are just slightly different, we may >consider to combine them. > > > aren't they just slightly different in just one structure member i.e. mr_poll and mr_send? >You may have seen that the transmit ring group and receive ring group >perform very different, so the mac_rx_ring_group_info_t should be very >different from mac_tx_ring_group_info_t. So we'd better to keep them >separately. > > ok. >In the MAC framework, we're using common data structures, >mac_ring_t/mac_ring_group_t, to simplify the resource management logic. > > > yeah, I see that and also mac_capab_ring_t is used on TX as well as RX side, that's I wondered why there is seaparate mac_rx_ring_info_t and mac_tx_ring_info_t and why not just mac_ring_info_t with a member mac_cb that can hold mac_poll for rx side rings and mac_send for TX side ring. >>4) >>AFAI understand these hardware resource capabilities can help do >>load balancing/packet classification , how it can help virtualization? >> >> >The ring grouping isolate VNICs from hardware layer, and the >virtualization is done at layer 2 instead of L3/L4. > Having user-prgrammed Rx rings(/flows) with L3/L4 classifications on top of VNIC/NIC, would differentiate crossbow features over other vendor's crossbow-like virtualization capabilities. It would be especially useful, when you want to run multiple applications such as streaming server(RTP), DHCP server (UDP)on same Virtual machine with a VNIC. such as no single app can monopolize given VNIC's already capped bandwidth. rx-ring grouping gives many benefits and I am not sure if we want to limit it to just VNIC/L2 classification. so I am trying to think-out loud here to wonder it's further exploitation in line with VNICs. >Ring group will be >the basic entity of virtualizing hardware resources. > > programming ring-group for L3/L4 classifications can help prformance of network virtual machines, I suppose. >>cause, As I see, virtual machines are identified using MAC+IP addresses, >>Is there any userland utilities that can help steer, classify and >>administer >>VM's traffic and steer across multiple rings by programming policy/rule on >>ring/s? >>Is so, what is it and how user can enforce a policy dynamically on given >>set of rings or ring groups? I know flowadm can program ring >>but can it program ring-group? >> >> >The updated Flow design will answer this. > > ok. :-) > > >>5) >>Can you group rings of different physical NICs, If yes, what is the >>interface for the same? >> >> >No. You may check out the ring group definition in the document. :) > > > so it's hardware design limitation of driver implementation limitation.? btw, I don't see the definition, can you help me with a page number? -Deepti >Thanks, > >Roamer > > >>-Deepti >> >> >>Yunsong (Roamer) Lu wrote: >> >> >> >>>Hi, >>>We've posted a new design document, Crossbow Hardware Resources >>>Management and Virtualization, here: >>>http://dlc.sun.com/osol/netvirt/downloads/docs/virtual_resources.pdf >>>you're welcome to review and comment. >>> >>>In this document, we introduce the new driver interfaces of virtualizing >>>NIC resources, like multiple rings and multiple MAC addresses, and we >>>talk about the ideas about organizing various hardware implementations >>>into the new framework. We're glad to know any comments from IHVs who >>>are designing new NICs. >>> >>>What's not included in the document? The new design evolves GLDv3 >>>interfaces, but before finalizing those changed interfaces, like >>>(*mc_tx)(), (*mc_unicast)(), (*mc_resoureces)(), mac_rx(), etc., we >>>would like to listen to your opinions first. Also, support for PCI-SIG >>>IOV will be added afterwards. >>> >>>Please feel free to comment and share your opinions! >>> >>>Thanks, >>> >>>Kais & Roamer >>> >>> >>> >>> >>> >>_______________________________________________ >>crossbow-discuss mailing list >>crossbow-discuss at opensolaris.org >>http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss >> >> > > >
