Hi Caitlin, Caitlin Bestler <mailto:[EMAIL PROTECTED]> wrote: > Some clarifications are needed here. > > First the Consumer is responsible for draining the > EVD after re-enabling it, or at least for remembering > that there may be undrained notified events.
Can you please explain what you mean by "re-enabling" the EVD ? Do you mean calling dat_evd_modify_upcall and changing the upcall policy from disable, back to enable ? > > That is "you-have-been-notified" is a sticky boolean > attribute that the Consumer is supposed to set to TRUE > when the upcall is made and only clear when the EVD > has been drained *after* re-enabling. > > Second, is that the EVD is first and foremost an event > *serializer*. It is presumed to have a finite number of > resources for making upcalls (at most one for the typical > case where SINGLE is enabled). The next upcall per > resource CANNOT occur until after the current upcall > has completed. > > Whether this should be solved in the DAT Provider is > a question of what the verb-layer provider is allowed > to do. If the verb layer provider can in fact generate > multiple concurrent upcalls for the same CQ then the > EVD itself must guard against re-entrancy. > > A more likely implementation is that upcalls triggered > by post_se, CM events and CQs could theoretically > occur at the same instance -- but that none of these > paths can be re-entrant by themselves. > > Once the potential re-entrancy from the verb layer > is known, then an optimal strategy can be selected. > For exaple, if the only potential re-entrancy comes > when the upcall interrupts a post_se call then some > simple critical regions can avoid all problems without > general purpose spinlocks or semaphores. > > On 8/16/05, James Lentini <[EMAIL PROTECTED]> wrote: >> >> >> On Tue, 16 Aug 2005, Guy German wrote: >> >>>>>>>>> Also, the pending_event_queue is only used for kDAPL generated >>>>>>>>> software events. This queue can be empty when there are >>>>>>>>> events on the CQ, so your would need to be expanded your >>>>>>>>> check to cover that. >>>>>>> >>>>>>> Actually, even though, I agreed before, I tend to disagree now. >>>>>>> The consumer will still get the DTO events as soon as the CQ >>>>>>> upcall is triggered (enabled), so only problem is with the >>>>>>> pending events list. >>>>>> >>>>>> Why is it an error for the consumer to modify the upcall policy >>>>>> when there are pending events? >>>>>> >>>>>> dat_evd_modify_upcall should behave just like the IBTA spec's >>>>>> Request Completion Notification verb in this respect. If there >>>>>> were events on the EVD before the upcall is enabled, no upcall >>>>>> needs to be generated. A correct consumer can easily work around >>>>>> this by enabling the upcall and polling the EVD one final time >>>>>> to ensure it is empty. >>>>> >>>>> There can be more than one event, and the consumer would need to >>>>> dequeue many times. While the consumer would do his extra >>>>> dequeue-ing he might also get an upcall, because his policy is >>>>> now enabled. I can't think of a design that can handle such a >>>>> case, and if there is one it is demanding and complicated, from >>>>> the consumers side. >>>> >>>> Isn't it the same position all event code written to the OpenIB >>>> API is in? >>> >>> I don't quite know what you are reffering to, but if you are >>> reffering to the case of cq in IB - It's totally different: you >>> only enable the cq once, so you will only get one upcall, and the >>> rest of the events you will need to dequeue. >> >> The consumer should only receive one upcall at a time if the upcall >> policy is DAT_UPCALL_SINGLE_INSTANCE. If the dequeues are performed >> in an upcall, the logic needed in an OpenIB consumer and kDAPL >> consumer is essentially the same. >> >> The difference is that the OpenIB consumer needs to re-enable the CQ >> upcall and poll to make sure no events were missed. >> >>>> I agree with you that this programming model is difficult to use, >>>> but I don't think it is impossible. >>> >>> I think it is a bad idea to dequeue events and at the same time >>> receive upcalls from the same queue. It is racy, and has bad >>> performance. I don't see *any* reason to do it. >> >> The current kDAPL implementation does create a situation in which an >> upcall and poll occur simultaneously if the upcall is disabled, the >> consumer enables the upcall, and then the consumer does a poll. In >> this scenario an upcall can occur while the consumer is polling. I >> was pointing out that this same race exists in the OpenIB verbs API >> (and the IBTA verbs). >> >> Again, I agree that we can eliminate the additional poll after >> enabling the upcall in kDAPL. We just need to do it in a way that is >> not hardware specific. I believe we can use the same technique we >> did in the DTO upcall. >> >> james >> _______________________________________________ >> openib-general mailing list >> [email protected] >> http://openib.org/mailman/listinfo/openib-general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
