03/04/2020 23:18, Morten Brørup: > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Jerin Jacob > > Thomas Monjalon <tho...@monjalon.net> wrote: > > > 02/04/2020 15:48, Andrzej Ostruszka [C]: > > > > On 3/26/20 6:42 PM, Andrzej Ostruszka wrote: > > > > > On 3/25/20 12:11 PM, Morten Brørup wrote: > > > > [...] > > > > >> And I am still strongly opposed to the callback method: > > > > > > > > > > Noted - however for now I would like to keep them. I don't have > > much > > > > > experience with this library so if they prove to be inadequate > > then we > > > > > will remove them. Right now they seem to add some flexibility > > that I like: > > > > > - if something should be changed globally and once (and it is > > safe to do > > > > > so!) then it can be done from the callback > > > > > - if something can be prepared once and consumed later by lcores > > then it > > > > > can be done in callback and the callback returns 0 so that > > event is > > > > > still queued and lcores (under assumption that queues are per > > lcore) > > > > > pick up what has been prepared. > > > > > > > > Morten > > > > > > > > I've been thinking about this a bit and would like to know your > > (and > > > > others) opinion about following proposed enhancement. > > > > > > > > Right now, how queues are used is left to the application decision > > (per > > > > lcore, per port, ...) - and I intend to keep it that way - but they > > are > > > > "match all". What I mean by that is that (unlike callbacks where > > you > > > > have separate per event type) queue has no chance to be selective. > > > > > > > > So if someone would like to go with queues only they he would have > > to > > > > coordinate between queues (or their "owners") which one does the > > > > handling of an event that supposedly should be handled only once. > > > > > > > > Let's take this forwarding example - the queues are per lcore and > > each > > > > lcore keeps its own copy of ARP table (hash) so when the change is > > > > noticed the event is queued to all registered queue, each lcore > > updates > > > > its own copy and everything is OK. However the routing is global > > (and > > > > right now is updated from callback) and if no callback is used for > > that > > > > then the event would be queued to all lcores and application would > > need > > > > to select the one which does the update. > > > > > > > > Would that be easier/better to register queue together with a > > bitmask of > > > > event types that given queue is accepting? Than during setup phase > > > > application would select just one queue to handle "global" events > > and > > > > the logic of event handling for lcores should be simplier. > > > > > > > > Let me know what you think. > > > > > > I think we want to avoid complicate design. > > > So let's choose between callback and message queue. > > > I vote for message queue because it can handle any situation, > > > and it allows to control the context of the event processing. > > > > IMO, it should be left to application decision, Application can use > > either callback or > > message queue based on their design and I don't think, DPDK needs to > > enforce certain model. > > On the upside, Giving two options, the application can choose the right > > model. > > The simple use case like updating the global routing table, The > > callback scheme would be more than enough. > > The downside of pushing the architecture to message queue would > > be that application either need to create additional control thread to > > poll or call select() > > get the event or in worst case check the message queue emptiness in > > fastpath. > > So why to enforce? > > > > Thoughts? > > A message queue would not require an additional control thread. It would use > the existing control thread that the application already has. > > I think you are missing an important point: > > The application needs to handle all control plane interactions, > not just control plane interactions related to the interface proxy library.
Yes this is the point. > So the application already has (or needs to add) mechanisms in place for > this. E.g. if a control plane event (from the interface proxy library or some > other trigger) needs to be distributed across a single or multiple data plane > lcores, the application already has (or needs to add) a mechanism for doing > it. Adding a specific mechanism only in this library does not help all the > other control plane interactions the application needs to handle. Actually it > does the opposite: it requires that the application handles events from the > interface proxy library in a specific way that is different from the way the > application already handles other control plane events. > > So I'm also voting for simplicity: A single event queue, leaving it up to the > application how to handle these events. > > > > The other reason is that I believe we need message queueing for > > > other purposes in DPDK (ex: multi-process, telemetry). > > > > As far as I know, telemetry is using Linux socket fro IPC, I am not > > sure > > why do we need to standardize message queue infra? Becasue, each use > > case is different. > > I think Thomas is suggesting that we consider the generic case of > interaction with the control plane, as I described above. > Not just interaction with the interface proxy events. > > > > > > > You start thinking about complex message management. > > > And I start thinking about other usages of message queueing. > > > So I think it is the right time to introduce a generic messaging in > > DPDK. > > > Note: the IPC rte_mp should be built on top of such generic > > messaging. > > > > > > If you agree, we can start a new email thread to better discuss > > > the generic messaging sub-system. > > I agree that it should be separated from the interface proxy library. > > And yes, DPDK is missing a generic framework - or at least a "best practices" > description - for interaction between the control plane and the data plane. > So far, every DPDK application developer has to come up with his own. > > > > I describe here the 3 properties I have in mind: > > > > > > 1/ Message policy > > > One very important rule in DPDK is to let the control to the > > application. > > > So the messaging policy must be managed by the application via DPDK > > API. > > > > Do you mean send() and recv() should be wrapped around DPDK call? I am thinking about something a bit more complex with handlers registration and default handlers in each DPDK library. > > > 2/ Message queue > > > It seems we should rely on ZeroMQ. Here is why: > > > http://zguide.zeromq.org/page:all#Why-We-Needed-ZeroMQ > > > > IMO, ZeroMQ used for IPC over network etc. In this case, the purpose is > > to pass the Netlink message IN THE SAME SYSTEM to application. > > Do you need external library dependency? On the same system or > > multiprocess application, our rte_ring would be more than enough. > > Right? > > If not, please enumerate the use case. Network communication will allow standardizing a DPDK remote control. With ZeroMQ, it comes for free. > > > 3/ Message format > > > I am not sure whether we can manage with "simple strings", TLV, > > > or should we use something more complex like protobuf? > > Lean and mean is the way to go. A binary format, please. > No more JSON or similar bloated encoding! JSON, as other text encoding as one advantage: it is readable when debugging. But I tend to agree that TLV is probably a good fit. > > In this use case, we are relying the Netlink message to application at > > least > > in Linux case. I think the message should be similar to Netlink message > > and give > > provision for other OS'es such as scheme. > > > > Why reinvent the wheel? I agree, we should not re-encode Netlink. With a TLV format, we can just encapsulate Netlink for the generic channel, and give it a message type to dispatch the message to the right hansler.
