On 02/20/2017 09:56 AM, Jan Blunck wrote: > On Fri, Feb 17, 2017 at 5:07 PM, Konrad Rzeszutek Wilk > <konrad.w...@oracle.com> wrote: >> On Thu, Feb 16, 2017 at 10:51:44PM +0100, Vincent JARDIN wrote: >>> Le 16/02/2017 à 14:36, Konrad Rzeszutek Wilk a écrit : >>>>> Is it time now to officially remove Dom0 support? >>>> So we do have an prototype implementation of netback but it is waiting >>>> for review of xen-devel to the spec. >>>> >>>> And I believe the implementation does utilize some of the dom0 >>>> parts of code in DPDK. >>> >>> Please, do you have URLs/pointers about it? It would be interesting to share >>> it with DPDK community too. >> >> Joao, would it be possible to include an tarball of the patches? I know >> they are no in the right state with the review of the staging >> grants API - they are incompatible, but it may help folks to get >> a feel for what DPDK APIs you used? >> >> Staging grants API: >> https://lists.xenproject.org/archives/html/xen-devel/2016-12/msg01878.html > > The topic of the grants API is unrelated to the dom0 memory pool. The > memory pool which uses xen_create_contiguous_region() is used in cases > we know that there are no hugepages available. Correct, I think what Konrad was trying to say was that xen-netback normally lives in a PV domain which doesn't have superpages, therefore such driver would need that memory pool part in order to work. The mentioned spec are additions to xen netif ABI for backend to safely map a fixed set of grant references (recycled overtime, provided by frontend) with the purpose of avoiding grant ops - DPDK would be one of the users.
> Joao and I met in Dublin and I whined about not being able to call > into the grants API from userspace and instead need to kick a kernel > driver to do the work for every burst. It would be great if that could > change in the future. Hm, I recall about that discussion. AFAIK you can do both grant alloc/revoke of pages through xengntshr_share_pages(...) and xengntshr_unshare(...) APIs provided by libxengnttab starting 4.7 or, libxc on older versions with xc_gntshr_share_pages/xc_gntshr_munmap. For the notification (or kicks) you can allocate the event channel in the guest with libevtchn starting 4.7, with xenevtchn_bind_unbound_port(...) or libxc on older versions with xc_evtchn_bind_unbound_port(...). And kick the guest with xenevtchn_notify or xc_evtchn_notify(...) [latter on older versions]. In short these APIs are ioctls to /dev/gntdev and /dev/evtchn. xenstore operations can also be done in userspace with libxenstore. To have the (similar) behavior of VRING_AVAIL_F_NO_INTERRUPT (i.e. avoiding the kicks) you "just" don't set rsp_event in ring (e.g. no calls to RING_FINAL_CHECK_FOR_RESPONSES), and keep checking for unconsumed Rx/Tx responses. For guest request notification (to wake up the backend for new Tx/Rx requests), you're dependent on whether backend requests it since it's the one setting req_event index. If it indeed sets it then you gotta use the evtchn notify that I depicted in the previous paragraph. Hope that helps! Joao  https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libs/gnttab/include/xengnttab.h;hb=HEAD  https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libs/evtchn/include/xenevtchn.h;hb=HEAD  https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxc/include/xenctrl_compat.h;hb=HEAD  https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/xenstore/include/xenstore.h;hb=HEAD