Cathy Zhou wrote: >> I am suggesting to use 2 streams. the shared lower stream will be used for >> passing down all dlpi control messages from all upper streams. this shared >> stream will be opened at dld_str_attach() time. when a upper stream sends a >> DL_BIND_REQ, this is when the 2nd lower stream (data path) is opened and >> bound to the specified sap. the upper stream would also exchange perstream >> info (rx/tx pointers) with the newly opened lower stream. >> >> so far we have: >> -a data path stream bound to a sap, it can send/receive data directly >> to/from dld. >> -a shared stream that's open but not ready to use since nothing has been >> sent to it yet. >> >> now, suppose we send a dlpi control message from the upper stream to nemo, >> nemo >> intercepts it and calls a mi_promisc/unicst/multicst entry point instead >> which >> would in turn send a corresponding dlpi message down the *shared* lower >> stream. >> >> the primitives listed above mostly affects shared mac state so its not that >> important which stream they get sent down from. if one (out of many) upper >> streams turns on promisc mode, the shared stream would start receiving >> packets as well. i_dls_link_rx_promisc() should do the right demultiplexing >> there. it could be made to pass packets only to promisc mode upper streams >> and >> drop everything else (because those would get passed up the data path >> lower stream instead). >> > [ As I'd like to see the possibility of the change, below I am just thinking > loudly. Please point me out if I am wrong. ] > > First, I think mi_unicst() has to be known by the data-path-stream, so that > the change of the unicast address wouldn't make that stream not be able to > receive any packets. >
I think today in nemo we have a MAC_NOTE_UNICST that updates the mac address of every dld_str_t/dls_impl_t. I know we can't assume the same for dlpi drivers so I think mi_unicst() should just send DL_SET_PHYS_ADDR_REQ down every data path stream and the shared stream. > Also, DLIOCRAW and DLIOCHDRINFO would be processes by both streams. Is that > right? > unfortunately yes. these would have to be sent down both streams. so on rx, it might be possible for an upper stream to get an M_DATA packet (with or without header) even though it did not ask for DLIOCRAW or DLIOCHDRINFO. in this case we would have to strip the header or construct a DL_UNITDATA_IND. > Leave that aside, if I understand correctly, "normal" packets would be > send/received from the data-path-stream, and "special" packets (received > because of the promiscuity-mode or the multicast packets) would be received > from the shared-lower-stream. > initially the data stream is the only stream that can receive packets (unicast + broadcast) since it's bound to a particular sap (e.g. IP). the shared stream is not bound so it cannot receive anything. if a upper stream issues a DL_ENABMULTI_REQ, this is when we have to bind the shared stream as well since we want multicast packets to go up the shared stream. but what sap do we bind to? it probably doesn't matter because we would have to turn on DL_PROMISC_SAP on the shared stream in order to get all multicast packets (not just ones destined to a particular sap). now we have two streams ready to receive data: data stream would receive unicast + broadcast packets. shared stream would recieve unicast + multicast + broadcast packets. both streams would enter softmac_rput() before nemo so some filtering could be done at that point. we could make unicast packets go only to the data path stream (via rxinfo->mpr_rx()) and broadcast + multicast go only to the shared stream (via mac_rx()). the check should require comparing only one byte. > That means every packets would go through both streams, and we would need to > change dls_accept() function and filter out the duplicated messages (the > policy to decide whether or not pass up to the DLS clients would become > complicated, see more below). supposing we do not turn on DL_PROMISC_PHYS on the lower stream (more later), why does dls_accept() need to change if we filter early in softmac as I mentioned above? dls_accept() will compare the per-dls_impl_t multicast list anyway so filtering is already being done. > At this point, without prototype, I am not > sure whether it can be done. The first concern I have is the performance > impact it may bring. > from mac and above, the performance impact is mostly likely small. what I am worried is what happens in the dlpi driver if we turn on DLS_PROMISC_SAP. If I remember correctly ce does dupmsg() when passing a packet up multiple streams right? if that's the case there shouldn't be much copying overhead. > Let's give an example, assume this is a softmac whose mac address is U1, and > the DLS client requests to add a multicast address M1. The broadcast address > is B1. Two lower streams would be set up - shared stream SS and data path > stream SD. On the receive side, packets of destination U1, M1, B1 would all > be passed to SS and in turn the i_dls_link_rx() function, and packets of > dest U1, B1 would also be passed to SD. We would filter out the U1, B1 > packets in SS to avoid duplicated messages. > the filtering could be done in softmac as explained above. > If the promiscuous mode is set, something similar would be done for the > shared-lower stream to filter out the duplicated messages. > to support DL_PROMISC_PHYS properly and at the same time get good data path performance might be hard. do you think it's ok to degrade data path performance when there are snoop streams open? if we can make this assumption we could just drop all packets coming from data path streams and let the demultiplexing get done properly by mac_rx() -> i_dls_link_rx_promisc() when packets arrive from the shared stream. > On the tx side, some special handling would also be needed. The packets > would be sent out either through the data-path-stream (normal case) or the > shared-lower-stream (promiscous mode case). We need to take great care, > which one to use, and especially when to qenable() the queue so dld_wsrv() > would be called. (Note that softmac_wsrv() of both lower streams might > ultimately result in a dld_wsrv(), an incorrect qenable() might cause the > packets disordering). I don't fully understand your tx path yet so I can't comment on how to do the flow control. but if I read the code correctly, when someone is snooping, the tx path doesn't really go through mac_txloop() but actually go does down to the dlpi driver and comes back up, right? also, I noticed you added a new function dld_wput_perstream_callback() and similar putnext() code in dld_wsrv(). the reason you do this here instead of within softmac_m_tx() is because you can't pass down the the lower stream write queue, right? if that's true, I think this is the right time to add a third argument to mi_tx(). crossbow will be adding that to support multiple tx channels. you can add this now to pass down the lower write queue. if this is done, the code within dld_wput_perstream_callback() could be moved to softmac_m_tx() and the code in dld_wsrv() could be removed, and mac_txloop() would work just like other drivers. >>>> 4. when promisc mode is turned on, packets will get delivered to both >>>> the data path stream (bound to a particular sap) and the shared lower >>>> stream. to prevent getting duplicate data, i_dls_link_rx_promisc() >>>> could be made to pass packets only to dls_impl_t's bound to >>>> DLS_PROMISC_SAP and drop everything else when there exists a separate >>>> lower data path stream. >>>> > > In another mail, you said the promiscuous mode is not turned on the > data-path-stream. Then how the packets can get delivered to both streams? > the data path stream is bound to a particular sap so it would only get unicast + broadcast packets destined to this sap. whereas the shared stream could have DL_PROMISC_* turned on and receive all packets sent to the data path stream + everything else. > If the promiscuous mode is turned on the data-path-stream, see above, > i_dls_link_rx_promsic() would have decode/compare each packets and pass up > all packets except those are destined to the MAC address (and the broadcast > address). > if you are willing to sacrifice data path performance when there are snoop streams open (as mentioned above), then no filtering needs to be done. we just blindly drop everything coming up from data path streams. >>> How about those multicst/broadcst packets? Do they come up from both >>> streams too? >> yes. they get passed up both streams. and the promisc mode shared stream >> would drop them. >> > Again. See above. > as suggested above, softmac would do the filtering: -broadcast packets would be dropped from all data streams -multicast packets would come up only from the shared stream (since we do not add multicast addresses to data streams) -unicast packets would be dropped from the shared stream >>> My current implementation basically follows the principle that shared >>> stream and per-stream stream are separated and serve for different purpose. >>> So they are independent from each other. Your proposal sounds more >>> complicated than that. But maybe I mis-understood your idea. >>> >> yes. I agree it's more complicated. but if the complication could make >> per-stream impact nemo less, or at least confine per-stream related code >> further, I think that's a good trade off. >> > Understood. But complication means it is more error-prone. Since this is the > late phase of the project, I also feel uncomfortable to make such a major > change. But I am still willing to make the changes, if the discussion turns > out this is a cleaner design. > I think we had a good discussion here. so even if you don't do this now due to time constraints, I think we have enough material to warrant some post-putback work. eric >> again, I am not insisting that this has to be done. I just want you to >> think through a bit whether doing this is worthwhile or not. There could >> be some cases that I missed that make this infeasible. >> > Thanks > - Cathy > > > _________________________________ > clearview-discuss mailing list > clearview-discuss at opensolaris.org
