On Mon, May 11, 2009 at 02:23:58PM -0700, Caitlin Bestler wrote: > On Thu, May 7, 2009 at 3:48 PM, Jason Gunthorpe > <[email protected]> wrote: > > > > Right, I was only thinking of a new driver call that was along the > > lines of update_mr_pages() that just updates the HCA's mapping with > > new page table entires atomically. It really would be device > > specific. If there is no call available then unregister/register + > > printk log is a fair generic implementation. > > > > To be clear, what I'm thinking is that this would only be invoked if > > Both the IBTA and RDMAC verbs were defined so that the meaning of > L-Key/R-Key/STag + Address could not instantly change from "X" to > "Y", only from "X" to NULL and then NULL to "Y".
Well, this is sort of a grey area, in one sense the meaning isn't changing, just the underlying phyiscal memory is being moved around by the OS. The notion that the verbs refer to some sort of invisible underlying VM object is nice for an implementation but pretty useless for MPI.. > There are a lot of good reasons for this, especially for R-Keys or > remotely accessible STags. It ensures that all operations that > started when the translation was "X" are completed before any that > will use the "Y" translation can commence. That is not something we > want to accidentally undermine. I'm not sure I see how this helps, synchronizing all this is the responsibility of the application, if it wants to change the mapping then it should be able to, and if it does so with poor timing then it will have races and loose data <shrug>. As it stands today there are already races where apps can loose data transfered after an unmap() or transfer the wrong data after a mmap() so the current model is already broken from that perspective. Of course an update verb has to operate with similar ordering guarantees to regsiter/unregister relative to the local work request queue - that is to say if the verb is done out-of-line with the WR queue then it must wait for the queue to flush before issuing the update to the HCA - just like unregister - and then wait for the verb to complete before returning to the app - just like register. And we all wish for userspace FRMRs... Jason _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
