Module Name: src
Committed By: martin
Date: Mon Jul 31 15:23:03 UTC 2023
Modified Files:
src/sys/arch/xen/include [netbsd-10]: hypervisor.h xenring.h
src/sys/arch/xen/x86 [netbsd-10]: cpu.c
src/sys/arch/xen/xen [netbsd-10]: if_xennet_xenbus.c xbd_xenbus.c
xbdback_xenbus.c xencons.c xengnt.c xennetback_xenbus.c
src/sys/arch/xen/xenbus [netbsd-10]: xenbus_comms.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #268):
sys/arch/xen/xenbus/xenbus_comms.c: revision 1.25
sys/arch/xen/xenbus/xenbus_comms.c: revision 1.26
sys/arch/xen/xen/xennetback_xenbus.c: revision 1.110
sys/arch/xen/xen/xennetback_xenbus.c: revision 1.111
sys/arch/xen/xen/xennetback_xenbus.c: revision 1.112
sys/arch/xen/x86/cpu.c: revision 1.144
sys/arch/xen/x86/cpu.c: revision 1.145
sys/arch/xen/include/hypervisor.h: revision 1.56
sys/arch/xen/include/hypervisor.h: revision 1.57
sys/arch/xen/xen/xbdback_xenbus.c: revision 1.102
sys/arch/xen/xen/xbdback_xenbus.c: revision 1.103
sys/arch/xen/include/xenring.h: revision 1.7
sys/arch/xen/xen/xennetback_xenbus.c: revision 1.109
sys/arch/xen/xen/xengnt.c: revision 1.40
sys/arch/xen/xen/xengnt.c: revision 1.41
sys/arch/xen/xen/if_xennet_xenbus.c: revision 1.129
sys/arch/xen/xen/xencons.c: revision 1.51
sys/arch/xen/xen/xencons.c: revision 1.52
sys/arch/xen/xen/xencons.c: revision 1.53
sys/arch/xen/xen/xbd_xenbus.c: revision 1.130 (patch)
sys/arch/xen/xen/xbd_xenbus.c: revision 1.131 (patch)
xen: Fix sense of xen_rmb/wmb to make sense.
Use membar_acquire and membar_release, not membar_consumer and
membar_producer, out of paranoia -- that better matches Linux's
rmb/wmb (at least for non-I/O loads and stores).
Proposed on port-xen:
https://mail-index.netbsd.org/port-xen/2022/07/13/msg010248.html
xen/x86/cpu.c: Membar audit.
I see no reason for store-before-load ordering here; as far as I'm
aware, evtchn_upcall_mask is only shared between a (v)CPU and its
(hypervisor) interrupts, not other (v)CPUs.
xennet(4): Membar audit.
- xennet_tx_complete: Other side owns rsp_prod, giving us responses
to tx commands. We own rsp_cons, recording which responess we've
processed already.
1. Other side initializes responses before advancing rsp_prod, so
we must observe rsp_prod before trying to examine the responses.
Hence load from rsp_prod must be followed by xen_rmb.
(Can this just use atomic_load_acquire?)
2. As soon as other side observes rsp_event, it may start to
overwrite now-unused response slots, so we must finish using the
response before advancing rsp_cons. Hence we must issue xen_wmb
before store to rsp_event.
(Can this just use atomic_store_release?)
(Should this use RING_FINAL_CHECK_FOR_RESPONSES?)
3. When loop is done and we set rsp_event, we must ensure the other
side has had a chance to see that we want more before we check
whether there is more to consume; otherwise the other side might
not bother to send us an interrupt. Hence after setting
rsp_event, we must issue xen_mb (store-before-load) before
re-checking rsp_prod.
- xennet_handler (rx): Same deal, except the xen_mb is buried in
RING_FINAL_CHECK_FOR_RESPONSES. Unclear why xennet_tx_complete has
this open-coded while xennet_handler (rx) uses the macro.
xbd(4): Membar audit.
After consuming slots, must issue xen_wmb before notifying the other
side that we've consumed them in RING_FINAL_CHECK_FOR_RESPONSES.
xbdback(4): Membar audit.
After consuming request slots, must issue xen_wmb notifying the other
side that we've consumed them in RING_FINAL_CHECK_FOR_REQUESTS.
xencons(4): Membar audit.
- xenconscn_getc: Once we have consumed an input slot, it is clearer
to issue xen_wmb (release, i.e., load/store-before-store) before
advancing in_cons so that the update becomes a store-release
freeing the input slot for the other side to reuse.
- xenconscn_putc: After filling an output slot, must issue xen_wmb
(release, i.e., load/store-before-store) before advancing out_prod,
and another one before notifying the other side of the advance.
xencons(4): Reduce unnecessary membars.
- xencons_handler: After advancing in_cons, only need one xen_wmb
before notifying the hypervisor that we're ready for more.
(XXX Should this do xen_mb and re-check in_prod at that point, or
does hypervisor_notify_via_evtchn obviate the need for this?)
- xenvonscn_getc: After reading in_prod, only need one xen_rmb before
using the slots it is telling us are now ready.
xengnt(4): Membar audit.
This had the sense of membars reversed, presumably because xen_rmb
and xen_wmb had gotten reversed at some point.
xenbus_comms.c: Membar audit.
This had the sense of membars reversed, presumably because xen_rmb
and xen_wmb had gotten reversed at some point.
xennetback(4): Fix xennetback_evthandler loop.
- After observing the other side has produced pending tx requests by
reading sring->req_prod, must issue xen_rmb before touching them.
Despite all the effort to use the heavy-weight
RING_FINAL_CHECK_FOR_REQUESTS on each request in the loop, this
barrier was missing.
- No need to update req_cons at each iteration in the loop. It's
private. Just update it once at the end.
- After consuming requests, must issue xen_wmb before releasing the
slots with RING_FINAL_CHECK_FOR_REQUEST for the other side to
reuse.
xennetback(4): Fix membars in xennetback_rx_copy_process.
- No need for barrier around touching req_cons and rsp_prod_pvt,
which are private.
- RING_PUSH_RESPONSES_AND_CHECK_NOTIFY already issues xen_wmb, no
need to add one explicitly.
- After pushing responses, must issue xen_wmb (not xen_rmb) before
hypervisor_notify_via_evtchn.
xennetback(4): Omit needless membars in xennetback_connect.
xneti is a private data structure to which we have exclusive access
here; ordering the stores doesn't make sense.
xen/hypervisor.h: Nix trailing whitespace.
No functional change intended.
xen/x86/cpu.c: Nix trailing whitespace.
No functional change intended.
xbd(4): Nix trailing whitespace.
xbdback(4): Nix trailing whitespace.
No functional change intended.
xencons(4): Nix trailing whitespace.
No functional change intended.
xengnt(4): Nix trailing whitespace.
No functional change intended.
xenbus_comms.c: Nix trailing whitespace.
No functional change intended.
xennetback(4): Nix trailing whitespace.
No functional change intended.
To generate a diff of this commit:
cvs rdiff -u -r1.55 -r1.55.4.1 src/sys/arch/xen/include/hypervisor.h
cvs rdiff -u -r1.6 -r1.6.20.1 src/sys/arch/xen/include/xenring.h
cvs rdiff -u -r1.142 -r1.142.4.1 src/sys/arch/xen/x86/cpu.c
cvs rdiff -u -r1.128 -r1.128.20.1 src/sys/arch/xen/xen/if_xennet_xenbus.c
cvs rdiff -u -r1.129.20.1 -r1.129.20.2 src/sys/arch/xen/xen/xbd_xenbus.c
cvs rdiff -u -r1.101 -r1.101.4.1 src/sys/arch/xen/xen/xbdback_xenbus.c
cvs rdiff -u -r1.50 -r1.50.20.1 src/sys/arch/xen/xen/xencons.c
cvs rdiff -u -r1.39 -r1.39.4.1 src/sys/arch/xen/xen/xengnt.c
cvs rdiff -u -r1.108 -r1.108.4.1 src/sys/arch/xen/xen/xennetback_xenbus.c
cvs rdiff -u -r1.24 -r1.24.20.1 src/sys/arch/xen/xenbus/xenbus_comms.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/arch/xen/include/hypervisor.h
diff -u src/sys/arch/xen/include/hypervisor.h:1.55 src/sys/arch/xen/include/hypervisor.h:1.55.4.1
--- src/sys/arch/xen/include/hypervisor.h:1.55 Wed Sep 7 00:40:19 2022
+++ src/sys/arch/xen/include/hypervisor.h Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: hypervisor.h,v 1.55 2022/09/07 00:40:19 knakahara Exp $ */
+/* $NetBSD: hypervisor.h,v 1.55.4.1 2023/07/31 15:23:02 martin Exp $ */
/*
* Copyright (c) 2006 Manuel Bouyer.
@@ -26,21 +26,21 @@
*/
/*
- *
+ *
* Communication to/from hypervisor.
- *
+ *
* Copyright (c) 2002-2004, K A Fraser
- *
+ *
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this source file (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use, copy, modify,
* merge, publish, distribute, sublicense, and/or sell copies of the Software,
* and to permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
- *
+ *
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
- *
+ *
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
@@ -111,8 +111,8 @@ struct xen_npx_attach_args {
#undef xen_wmb
#define xen_mb() membar_sync()
-#define xen_rmb() membar_producer()
-#define xen_wmb() membar_consumer()
+#define xen_rmb() membar_acquire()
+#define xen_wmb() membar_release()
#endif /* __XEN_INTERFACE_VERSION */
#include <machine/xen/hypercalls.h>
@@ -187,10 +187,10 @@ void hypervisor_set_ipending(uint64_t, i
void hypervisor_machdep_attach(void);
void hypervisor_machdep_resume(void);
-/*
+/*
* Force a proper event-channel callback from Xen after clearing the
* callback mask. We do this in a very simple manner, by making a call
- * down into Xen. The pending flag will be checked by Xen on return.
+ * down into Xen. The pending flag will be checked by Xen on return.
*/
static __inline void hypervisor_force_callback(void)
{
Index: src/sys/arch/xen/include/xenring.h
diff -u src/sys/arch/xen/include/xenring.h:1.6 src/sys/arch/xen/include/xenring.h:1.6.20.1
--- src/sys/arch/xen/include/xenring.h:1.6 Sat Apr 25 15:26:17 2020
+++ src/sys/arch/xen/include/xenring.h Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: xenring.h,v 1.6 2020/04/25 15:26:17 bouyer Exp $ */
+/* $NetBSD: xenring.h,v 1.6.20.1 2023/07/31 15:23:02 martin Exp $ */
/*
* Glue goop for xbd ring request/response protocol structures.
@@ -25,8 +25,8 @@
#undef xen_wmb
#define xen_mb() membar_sync()
-#define xen_rmb() membar_producer()
-#define xen_wmb() membar_consumer()
+#define xen_rmb() membar_acquire()
+#define xen_wmb() membar_release()
/*
* Define ring types. These were previously part of the public API.
Index: src/sys/arch/xen/x86/cpu.c
diff -u src/sys/arch/xen/x86/cpu.c:1.142 src/sys/arch/xen/x86/cpu.c:1.142.4.1
--- src/sys/arch/xen/x86/cpu.c:1.142 Sat Aug 20 23:48:51 2022
+++ src/sys/arch/xen/x86/cpu.c Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: cpu.c,v 1.142 2022/08/20 23:48:51 riastradh Exp $ */
+/* $NetBSD: cpu.c,v 1.142.4.1 2023/07/31 15:23:02 martin Exp $ */
/*-
* Copyright (c) 2000 The NetBSD Foundation, Inc.
@@ -65,7 +65,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: cpu.c,v 1.142 2022/08/20 23:48:51 riastradh Exp $");
+__KERNEL_RCSID(0, "$NetBSD: cpu.c,v 1.142.4.1 2023/07/31 15:23:02 martin Exp $");
#include "opt_ddb.h"
#include "opt_multiprocessor.h"
@@ -521,7 +521,7 @@ cpu_attach_common(device_t parent, devic
(void *)pcb->pcb_rsp
#endif
);
-
+
}
#endif /* MPVERBOSE */
}
@@ -690,10 +690,10 @@ cpu_boot_secondary(struct cpu_info *ci)
/*
* APs end up here immediately after initialisation and VCPUOP_up in
- * mp_cpu_start().
+ * mp_cpu_start().
* At this point, we are running in the idle pcb/idle stack of the new
* CPU. This function jumps to the idle loop and starts looking for
- * work.
+ * work.
*/
extern void x86_64_tls_switch(struct lwp *);
void
@@ -848,7 +848,7 @@ xen_init_amd64_vcpuctxt(struct cpu_info
/* resume with interrupts off */
vci = ci->ci_vcpu;
vci->evtchn_upcall_mask = 1;
- xen_mb();
+ __insn_barrier();
/* resume in kernel-mode */
initctx->flags = VGCF_in_kernel | VGCF_online;
@@ -931,8 +931,8 @@ xen_init_i386_vcpuctxt(struct cpu_info *
gdt_prepframes(frames, (vaddr_t)ci->ci_gdt, gdt_ents);
- /*
- * Initialise the vcpu context:
+ /*
+ * Initialise the vcpu context:
* We use this cpu's idle_loop() pcb context.
*/
@@ -945,7 +945,7 @@ xen_init_i386_vcpuctxt(struct cpu_info *
/* resume with interrupts off */
vci = ci->ci_vcpu;
vci->evtchn_upcall_mask = 1;
- xen_mb();
+ __insn_barrier();
/* resume in kernel-mode */
initctx->flags = VGCF_in_kernel | VGCF_online;
@@ -1159,7 +1159,7 @@ cpu_load_pmap(struct pmap *pmap, struct
/*
* pmap_cpu_init_late: perform late per-CPU initialization.
- *
+ *
* Short note about percpu PDIR pages. Both the PAE and __x86_64__ architectures
* have per-cpu PDIR tables, for two different reasons:
* - on PAE, this is to get around Xen's pagetable setup constraints (multiple
@@ -1169,7 +1169,7 @@ cpu_load_pmap(struct pmap *pmap, struct
* (see cpu_load_pmap()).
*
* What this means for us is that the PDIR of the pmap_kernel() is considered
- * to be a canonical "SHADOW" PDIR with the following properties:
+ * to be a canonical "SHADOW" PDIR with the following properties:
* - its recursive mapping points to itself
* - per-cpu recursive mappings point to themselves on __x86_64__
* - per-cpu L4 pages' kernel entries are expected to be in sync with
Index: src/sys/arch/xen/xen/if_xennet_xenbus.c
diff -u src/sys/arch/xen/xen/if_xennet_xenbus.c:1.128 src/sys/arch/xen/xen/if_xennet_xenbus.c:1.128.20.1
--- src/sys/arch/xen/xen/if_xennet_xenbus.c:1.128 Wed Aug 26 15:54:10 2020
+++ src/sys/arch/xen/xen/if_xennet_xenbus.c Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: if_xennet_xenbus.c,v 1.128 2020/08/26 15:54:10 riastradh Exp $ */
+/* $NetBSD: if_xennet_xenbus.c,v 1.128.20.1 2023/07/31 15:23:02 martin Exp $ */
/*
* Copyright (c) 2006 Manuel Bouyer.
@@ -81,7 +81,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: if_xennet_xenbus.c,v 1.128 2020/08/26 15:54:10 riastradh Exp $");
+__KERNEL_RCSID(0, "$NetBSD: if_xennet_xenbus.c,v 1.128.20.1 2023/07/31 15:23:02 martin Exp $");
#include "opt_xen.h"
#include "opt_nfs_boot.h"
@@ -951,12 +951,12 @@ again:
SLIST_INSERT_HEAD(&sc->sc_txreq_head, req, txreq_next);
sc->sc_free_txreql++;
}
-
sc->sc_tx_ring.rsp_cons = resp_prod;
/* set new event and check for race with rsp_cons update */
+ xen_wmb();
sc->sc_tx_ring.sring->rsp_event =
resp_prod + ((sc->sc_tx_ring.sring->req_prod - resp_prod) >> 1) + 1;
- xen_wmb();
+ xen_mb();
if (resp_prod != sc->sc_tx_ring.sring->rsp_prod)
goto again;
}
@@ -1060,8 +1060,8 @@ again:
if_statinc(ifp, if_iqdrops);
m_freem(m0);
}
- xen_rmb();
sc->sc_rx_ring.rsp_cons = i;
+ xen_wmb();
RING_FINAL_CHECK_FOR_RESPONSES(&sc->sc_rx_ring, more_to_do);
mutex_exit(&sc->sc_rx_lock);
Index: src/sys/arch/xen/xen/xbd_xenbus.c
diff -u src/sys/arch/xen/xen/xbd_xenbus.c:1.129.20.1 src/sys/arch/xen/xen/xbd_xenbus.c:1.129.20.2
--- src/sys/arch/xen/xen/xbd_xenbus.c:1.129.20.1 Thu Jul 27 17:56:31 2023
+++ src/sys/arch/xen/xen/xbd_xenbus.c Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: xbd_xenbus.c,v 1.129.20.1 2023/07/27 17:56:31 martin Exp $ */
+/* $NetBSD: xbd_xenbus.c,v 1.129.20.2 2023/07/31 15:23:02 martin Exp $ */
/*
* Copyright (c) 2006 Manuel Bouyer.
@@ -50,7 +50,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: xbd_xenbus.c,v 1.129.20.1 2023/07/27 17:56:31 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: xbd_xenbus.c,v 1.129.20.2 2023/07/31 15:23:02 martin Exp $");
#include "opt_xen.h"
@@ -99,7 +99,7 @@ __KERNEL_RCSID(0, "$NetBSD: xbd_xenbus.c
#define XBD_XFER_LIMIT (2*XBD_MAX_XFER)
#define XEN_BSHIFT 9 /* log2(XEN_BSIZE) */
-#define XEN_BSIZE (1 << XEN_BSHIFT)
+#define XEN_BSIZE (1 << XEN_BSHIFT)
CTASSERT((MAXPHYS <= 2*XBD_MAX_CHUNK));
CTASSERT(XEN_BSIZE == DEV_BSIZE);
@@ -676,7 +676,7 @@ xbd_backend_changed(void *arg, XenbusSta
sc->sc_xbdsize =
sc->sc_sectors * (uint64_t)sc->sc_secsize / DEV_BSIZE;
dg = &sc->sc_dksc.sc_dkdev.dk_geom;
- memset(dg, 0, sizeof(*dg));
+ memset(dg, 0, sizeof(*dg));
dg->dg_secperunit = sc->sc_sectors;
dg->dg_secsize = sc->sc_secsize;
@@ -735,25 +735,26 @@ xbd_connect(struct xbd_xenbus_softc *sc)
err = xenbus_read_ul(NULL,
sc->sc_xbusd->xbusd_path, "virtual-device", &sc->sc_handle, 10);
if (err)
- panic("%s: can't read number from %s/virtual-device\n",
+ panic("%s: can't read number from %s/virtual-device\n",
device_xname(sc->sc_dksc.sc_dev),
sc->sc_xbusd->xbusd_otherend);
err = xenbus_read_ul(NULL,
sc->sc_xbusd->xbusd_otherend, "info", &sc->sc_info, 10);
if (err)
- panic("%s: can't read number from %s/info\n",
+ panic("%s: can't read number from %s/info\n",
device_xname(sc->sc_dksc.sc_dev),
sc->sc_xbusd->xbusd_otherend);
err = xenbus_read_ul(NULL,
sc->sc_xbusd->xbusd_otherend, "sector-size", &sc->sc_secsize, 10);
if (err)
- panic("%s: can't read number from %s/sector-size\n",
+ panic("%s: can't read number from %s/sector-size\n",
device_xname(sc->sc_dksc.sc_dev),
sc->sc_xbusd->xbusd_otherend);
+
err = xenbus_read_ull(NULL,
sc->sc_xbusd->xbusd_otherend, "sectors", §ors, 10);
if (err)
- panic("%s: can't read number from %s/sectors\n",
+ panic("%s: can't read number from %s/sectors\n",
device_xname(sc->sc_dksc.sc_dev),
sc->sc_xbusd->xbusd_otherend);
sc->sc_sectors = sectors * (uint64_t)XEN_BSIZE / sc->sc_secsize;
@@ -935,7 +936,7 @@ xbd_iosize(device_t dev, int *maxxfer)
{
/*
* Always restrict dumps to XBD_MAX_XFER to avoid indirect segments,
- * so that it uses as little memory as possible.
+ * so that it uses as little memory as possible.
*/
if (*maxxfer > XBD_MAX_XFER)
*maxxfer = XBD_MAX_XFER;
@@ -1009,7 +1010,7 @@ xbdsize(dev_t dev)
static int
xbdread(dev_t dev, struct uio *uio, int flags)
{
- struct xbd_xenbus_softc *sc =
+ struct xbd_xenbus_softc *sc =
device_lookup_private(&xbd_cd, DISKUNIT(dev));
struct dk_softc *dksc = &sc->sc_dksc;
@@ -1282,7 +1283,7 @@ xbd_diskstart_submit(struct xbd_xenbus_s
__func__, req->id, req->operation, req->sector_number,
req->handle));
- size = uimin(bp->b_bcount - start, XBD_MAX_CHUNK);
+ size = uimin(bp->b_bcount - start, XBD_MAX_CHUNK);
for (dmaseg = 0; dmaseg < dmamap->dm_nsegs && size > 0; dmaseg++) {
bus_dma_segment_t *ds = &dmamap->dm_segs[dmaseg];
Index: src/sys/arch/xen/xen/xbdback_xenbus.c
diff -u src/sys/arch/xen/xen/xbdback_xenbus.c:1.101 src/sys/arch/xen/xen/xbdback_xenbus.c:1.101.4.1
--- src/sys/arch/xen/xen/xbdback_xenbus.c:1.101 Thu Sep 1 15:33:23 2022
+++ src/sys/arch/xen/xen/xbdback_xenbus.c Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: xbdback_xenbus.c,v 1.101 2022/09/01 15:33:23 bouyer Exp $ */
+/* $NetBSD: xbdback_xenbus.c,v 1.101.4.1 2023/07/31 15:23:02 martin Exp $ */
/*
* Copyright (c) 2006 Manuel Bouyer.
@@ -26,7 +26,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: xbdback_xenbus.c,v 1.101 2022/09/01 15:33:23 bouyer Exp $");
+__KERNEL_RCSID(0, "$NetBSD: xbdback_xenbus.c,v 1.101.4.1 2023/07/31 15:23:02 martin Exp $");
#include <sys/buf.h>
#include <sys/condvar.h>
@@ -105,7 +105,7 @@ typedef enum {WAITING, RUN, DISCONNECTIN
* event channels, biointr() soft interrupts, xenbus commands), the xbdi_lock
* mutex is used to protect specific elements of the xbdback instance from
* concurrent access: thread status and ring access (when pushing responses).
- *
+ *
* Here's how the call graph is supposed to be for a single I/O:
*
* xbdback_co_main()
@@ -125,7 +125,7 @@ typedef enum {WAITING, RUN, DISCONNECTIN
* | |
* | xbdback_co_main_incr() -> xbdback_co_main_loop()
* |
- * xbdback_co_do_io()
+ * xbdback_co_do_io()
* |
* xbdback_co_main_incr() -> xbdback_co_main_loop()
*/
@@ -210,7 +210,7 @@ struct xbdback_instance {
vaddr_t xbdi_ring_va; /* to unmap the ring */
/* disconnection must be postponed until all I/O is done */
int xbdi_refcnt;
- /*
+ /*
* State for I/O processing/coalescing follows; this has to
* live here instead of on the stack because of the
* continuation-ness (see above).
@@ -361,7 +361,7 @@ xbdback_xenbus_create(struct xenbus_devi
mutex_init(&xbdi->xbdi_lock, MUTEX_DEFAULT, IPL_BIO);
cv_init(&xbdi->xbdi_cv, xbdi->xbdi_name);
- xbusd->xbusd_u.b.b_cookie = xbdi;
+ xbusd->xbusd_u.b.b_cookie = xbdi;
xbusd->xbusd_u.b.b_detach = xbdback_xenbus_destroy;
xbusd->xbusd_otherend_changed = xbdback_frontend_changed;
xbdi->xbdi_xbusd = xbusd;
@@ -615,7 +615,7 @@ err1:
static void
xbdback_disconnect(struct xbdback_instance *xbdi)
{
-
+
mutex_enter(&xbdi->xbdi_lock);
if (xbdi->xbdi_status == DISCONNECTED) {
mutex_exit(&xbdi->xbdi_lock);
@@ -759,7 +759,7 @@ xbdback_backend_changed(struct xenbus_wa
/* If both Ioctls failed set device size to 0 and return */
printf("xbdback %s: can't DIOCGWEDGEINFO device "
"0x%"PRIx64": %d\n", xbusd->xbusd_path,
- xbdi->xbdi_dev, err);
+ xbdi->xbdi_dev, err);
xbdi->xbdi_size = xbdi->xbdi_dev = 0;
vn_close(xbdi->xbdi_vp, FREAD, NOCRED);
xbdi->xbdi_vp = NULL;
@@ -901,7 +901,7 @@ xbdback_thread(void *arg)
cv_wait(&xbdi->xbdi_cv, &xbdi->xbdi_lock);
continue;
}
-
+
/* All I/Os should have been processed by now,
* xbdi_refcnt should drop to 0 */
xbdi_put(xbdi);
@@ -941,7 +941,7 @@ xbdback_co_main(struct xbdback_instance
* the ring.
*/
static void *
-xbdback_co_main_loop(struct xbdback_instance *xbdi, void *obj __unused)
+xbdback_co_main_loop(struct xbdback_instance *xbdi, void *obj __unused)
{
blkif_request_t *req, *reqn;
blkif_x86_32_request_t *req32;
@@ -1065,6 +1065,7 @@ xbdback_co_main_done2(struct xbdback_ins
{
int work_to_do;
+ xen_wmb();
RING_FINAL_CHECK_FOR_REQUESTS(&xbdi->xbdi_ring.ring_n, work_to_do);
if (work_to_do)
xbdi->xbdi_cont = xbdback_co_main;
@@ -1114,7 +1115,7 @@ xbdback_co_cache_doflush(struct xbdback_
*/
static void *
xbdback_co_io(struct xbdback_instance *xbdi, void *obj __unused)
-{
+{
int i, error;
blkif_request_t *req, *reqn;
blkif_x86_32_request_t *req32;
@@ -1241,7 +1242,7 @@ xbdback_co_io_gotio(struct xbdback_insta
xbdi_get(xbdi);
xbdi->xbdi_pendingreqs++;
-
+
req = &xbdi->xbdi_xen_req;
xbd_io = obj;
memset(xbd_io, 0, sizeof(*xbd_io));
@@ -1431,7 +1432,7 @@ xbdback_iodone_locked(struct xbdback_ins
status = BLKIF_RSP_ERROR;
} else
status = BLKIF_RSP_OKAY;
-
+
xbdback_send_reply(xbdi, xbd_io->xio_id, xbd_io->xio_operation, status);
xbdi_put(xbdi);
@@ -1535,7 +1536,7 @@ xbdback_map_shm(struct xbdback_io *xbd_i
xbd_io->xio_vaddr = xbd_io->xio_xv->xv_vaddr;
error = xen_shm_map(xbd_io->xio_nrma, xbdi->xbdi_domid,
- xbd_io->xio_gref, xbd_io->xio_vaddr, xbd_io->xio_gh,
+ xbd_io->xio_gref, xbd_io->xio_vaddr, xbd_io->xio_gh,
(xbd_io->xio_operation == BLKIF_OP_WRITE) ? XSHM_RO : 0);
switch(error) {
Index: src/sys/arch/xen/xen/xencons.c
diff -u src/sys/arch/xen/xen/xencons.c:1.50 src/sys/arch/xen/xen/xencons.c:1.50.20.1
--- src/sys/arch/xen/xen/xencons.c:1.50 Thu May 7 19:25:57 2020
+++ src/sys/arch/xen/xen/xencons.c Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: xencons.c,v 1.50 2020/05/07 19:25:57 maxv Exp $ */
+/* $NetBSD: xencons.c,v 1.50.20.1 2023/07/31 15:23:02 martin Exp $ */
/*
* Copyright (c) 2006 Manuel Bouyer.
@@ -53,7 +53,7 @@
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: xencons.c,v 1.50 2020/05/07 19:25:57 maxv Exp $");
+__KERNEL_RCSID(0, "$NetBSD: xencons.c,v 1.50.20.1 2023/07/31 15:23:02 martin Exp $");
#include "opt_xen.h"
@@ -82,10 +82,10 @@ __KERNEL_RCSID(0, "$NetBSD: xencons.c,v
#endif
#undef XENDEBUG
-
+
#ifdef XENDEBUG
#define XENPRINTK(x) printk x
-#else
+#else
#define XENPRINTK(x)
#endif
@@ -127,7 +127,7 @@ const struct cdevsw xencons_cdevsw = {
.d_close = xencons_close,
.d_read = xencons_read,
.d_write = xencons_write,
- .d_ioctl = xencons_ioctl,
+ .d_ioctl = xencons_ioctl,
.d_stop = xencons_stop,
.d_tty = xencons_tty,
.d_poll = xencons_poll,
@@ -328,7 +328,7 @@ xencons_poll(dev_t dev, int events, stru
struct xencons_softc *sc = device_lookup_private(&xencons_cd,
XENCONS_UNIT(dev));
struct tty *tp = sc->sc_tty;
-
+
return ((*tp->t_linesw->l_poll)(tp, events, l));
}
@@ -442,7 +442,7 @@ xencons_stop(struct tty *tp, int flag)
}
/* Non-privileged console interrupt routine */
-static int
+static int
xencons_handler(void *arg)
{
struct xencons_softc *sc = arg;
@@ -453,7 +453,7 @@ xencons_handler(void *arg)
splx(s);
return 1;
}
-
+
#define XNC_IN (xencons_interface->in)
@@ -479,9 +479,9 @@ xencons_handler(void *arg)
cons += len;
xen_wmb();
xencons_interface->in_cons = cons;
- xen_wmb();
}
}
+ xen_wmb();
hypervisor_notify_via_evtchn(xen_start_info.console.domU.evtchn);
splx(s);
return 1;
@@ -575,7 +575,6 @@ xenconscn_getc(dev_t dev)
cons = xencons_interface->in_cons;
prod = xencons_interface->in_prod;
- xen_rmb();
while (cons == prod) {
HYPERVISOR_yield();
prod = xencons_interface->in_prod;
@@ -583,7 +582,7 @@ xenconscn_getc(dev_t dev)
xen_rmb();
c = xencons_interface->in[MASK_XENCONS_IDX(xencons_interface->in_cons,
xencons_interface->in)];
- xen_rmb();
+ xen_wmb();
xencons_interface->in_cons = cons + 1;
cn_check_magic(dev, c, xencons_cnm_state);
splx(s);
@@ -614,9 +613,9 @@ xenconscn_putc(dev_t dev, int c)
}
xencons_interface->out[MASK_XENCONS_IDX(xencons_interface->out_prod,
xencons_interface->out)] = c;
- xen_rmb();
+ xen_wmb();
xencons_interface->out_prod++;
- xen_rmb();
+ xen_wmb();
hypervisor_notify_via_evtchn(xen_start_info.console.domU.evtchn);
splx(s);
}
Index: src/sys/arch/xen/xen/xengnt.c
diff -u src/sys/arch/xen/xen/xengnt.c:1.39 src/sys/arch/xen/xen/xengnt.c:1.39.4.1
--- src/sys/arch/xen/xen/xengnt.c:1.39 Fri Jun 3 10:42:17 2022
+++ src/sys/arch/xen/xen/xengnt.c Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: xengnt.c,v 1.39 2022/06/03 10:42:17 bouyer Exp $ */
+/* $NetBSD: xengnt.c,v 1.39.4.1 2023/07/31 15:23:02 martin Exp $ */
/*
* Copyright (c) 2006 Manuel Bouyer.
@@ -26,7 +26,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: xengnt.c,v 1.39 2022/06/03 10:42:17 bouyer Exp $");
+__KERNEL_RCSID(0, "$NetBSD: xengnt.c,v 1.39.4.1 2023/07/31 15:23:02 martin Exp $");
#include <sys/types.h>
#include <sys/param.h>
@@ -212,7 +212,7 @@ xengnt_suspend(void) {
/* invalidate all grant entries (necessary for resume) */
gnt_entries[i] = XENGNT_NO_ENTRY;
}
-
+
/* Remove virtual => machine mapping for grant table */
pmap_kremove((vaddr_t)grant_table.gntt, gnt_nr_grant_frames * PAGE_SIZE);
@@ -450,21 +450,21 @@ xengnt_grant_access(domid_t dom, paddr_t
grant_table.gntt_v2[*entryp].full_page.frame = ma >> PAGE_SHIFT;
grant_table.gntt_v2[*entryp].hdr.domid = dom;
/*
- * ensure that the above values reach global visibility
+ * ensure that the above values reach global visibility
* before permitting frame's access (done when we set flags)
*/
- xen_rmb();
+ xen_wmb();
grant_table.gntt_v2[*entryp].hdr.flags =
GTF_permit_access | (ro ? GTF_readonly : 0);
} else {
grant_table.gntt_v1[*entryp].frame = ma >> PAGE_SHIFT;
grant_table.gntt_v1[*entryp].domid = dom;
- /*
+ /*
* ensure that the above values reach global visibility
- * before permitting frame's access (done when we set flags)
+ * before permitting frame's access (done when we set flags)
*/
- xen_rmb();
- grant_table.gntt_v1[*entryp].flags =
+ xen_wmb();
+ grant_table.gntt_v1[*entryp].flags =
GTF_permit_access | (ro ? GTF_readonly : 0);
}
mutex_exit(&grant_lock);
@@ -472,7 +472,7 @@ xengnt_grant_access(domid_t dom, paddr_t
}
static inline uint16_t
-xen_atomic_cmpxchg16(volatile uint16_t *ptr, uint16_t val, uint16_t newval)
+xen_atomic_cmpxchg16(volatile uint16_t *ptr, uint16_t val, uint16_t newval)
{
unsigned long result;
Index: src/sys/arch/xen/xen/xennetback_xenbus.c
diff -u src/sys/arch/xen/xen/xennetback_xenbus.c:1.108 src/sys/arch/xen/xen/xennetback_xenbus.c:1.108.4.1
--- src/sys/arch/xen/xen/xennetback_xenbus.c:1.108 Fri Sep 2 23:48:10 2022
+++ src/sys/arch/xen/xen/xennetback_xenbus.c Mon Jul 31 15:23:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: xennetback_xenbus.c,v 1.108 2022/09/02 23:48:10 thorpej Exp $ */
+/* $NetBSD: xennetback_xenbus.c,v 1.108.4.1 2023/07/31 15:23:02 martin Exp $ */
/*
* Copyright (c) 2006 Manuel Bouyer.
@@ -25,7 +25,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: xennetback_xenbus.c,v 1.108 2022/09/02 23:48:10 thorpej Exp $");
+__KERNEL_RCSID(0, "$NetBSD: xennetback_xenbus.c,v 1.108.4.1 2023/07/31 15:23:02 martin Exp $");
#include <sys/types.h>
#include <sys/param.h>
@@ -117,7 +117,7 @@ struct xnetback_instance {
grant_handle_t xni_tx_ring_handle; /* to unmap the ring */
grant_handle_t xni_rx_ring_handle;
vaddr_t xni_tx_ring_va; /* to unmap the ring */
- vaddr_t xni_rx_ring_va;
+ vaddr_t xni_rx_ring_va;
/* arrays used in xennetback_ifstart(), used for both Rx and Tx */
gnttab_copy_t xni_gop_copy[NB_XMIT_PAGES_BATCH];
@@ -497,9 +497,7 @@ xennetback_connect(struct xnetback_insta
goto err2;
}
xneti->xni_evtchn = evop.u.bind_interdomain.local_port;
- xen_wmb();
xneti->xni_status = CONNECTED;
- xen_wmb();
xneti->xni_ih = xen_intr_establish_xname(-1, &xen_pic,
xneti->xni_evtchn, IST_LEVEL, IPL_NET, xennetback_evthandler,
@@ -784,7 +782,7 @@ xennetback_tx_m0len_fragment(struct xnet
{
netif_tx_request_t *txreq;
- /* This assumes all the requests are already pushed into the ring */
+ /* This assumes all the requests are already pushed into the ring */
*cntp = 1;
do {
txreq = RING_GET_REQUEST(&xneti->xni_txring, req_cons);
@@ -805,7 +803,7 @@ xennetback_evthandler(void *arg)
netif_tx_request_t txreq;
struct mbuf *m, *m0 = NULL, *mlast = NULL;
int receive_pending;
- RING_IDX req_cons;
+ RING_IDX req_cons, req_prod;
int queued = 0, m0_len = 0;
struct xnetback_xstate *xst;
const bool discard = ((ifp->if_flags & (IFF_UP | IFF_RUNNING)) !=
@@ -813,17 +811,12 @@ xennetback_evthandler(void *arg)
XENPRINTF(("xennetback_evthandler "));
req_cons = xneti->xni_txring.req_cons;
- while (1) {
- xen_rmb(); /* be sure to read the request before updating */
- xneti->xni_txring.req_cons = req_cons;
- xen_wmb();
- RING_FINAL_CHECK_FOR_REQUESTS(&xneti->xni_txring,
- receive_pending);
- if (receive_pending == 0)
- break;
+again:
+ req_prod = xneti->xni_txring.sring->req_prod;
+ xen_rmb();
+ while (req_cons != req_prod) {
RING_COPY_REQUEST(&xneti->xni_txring, req_cons,
&txreq);
- xen_rmb();
XENPRINTF(("%s pkt size %d\n", xneti->xni_if.if_xname,
txreq.size));
req_cons++;
@@ -962,6 +955,12 @@ mbuf_fail:
queued = 0;
}
}
+ xen_wmb();
+ RING_FINAL_CHECK_FOR_REQUESTS(&xneti->xni_txring, receive_pending);
+ if (receive_pending)
+ goto again;
+ xneti->xni_txring.req_cons = req_cons;
+
if (m0) {
/* Queue empty, and still unfinished multi-fragment request */
printf("%s: dropped unfinished multi-fragment\n",
@@ -1021,14 +1020,13 @@ xennetback_rx_copy_process(struct ifnet
}
/* update pointer */
- xen_rmb();
xneti->xni_rxring.req_cons += queued;
xneti->xni_rxring.rsp_prod_pvt += queued;
RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&xneti->xni_rxring, notify);
/* send event */
if (notify) {
- xen_rmb();
+ xen_wmb();
XENPRINTF(("%s receive event\n",
xneti->xni_if.if_xname));
hypervisor_notify_via_evtchn(xneti->xni_evtchn);
@@ -1271,7 +1269,7 @@ again:
* here, as the frontend doesn't notify when adding
* requests anyway
*/
- if (__predict_false(abort ||
+ if (__predict_false(abort ||
!RING_HAS_UNCONSUMED_REQUESTS(&xneti->xni_rxring))) {
/* ring full */
ifp->if_timer = 1;
Index: src/sys/arch/xen/xenbus/xenbus_comms.c
diff -u src/sys/arch/xen/xenbus/xenbus_comms.c:1.24 src/sys/arch/xen/xenbus/xenbus_comms.c:1.24.20.1
--- src/sys/arch/xen/xenbus/xenbus_comms.c:1.24 Wed May 13 13:19:38 2020
+++ src/sys/arch/xen/xenbus/xenbus_comms.c Mon Jul 31 15:23:02 2023
@@ -1,24 +1,24 @@
-/* $NetBSD: xenbus_comms.c,v 1.24 2020/05/13 13:19:38 jdolecek Exp $ */
+/* $NetBSD: xenbus_comms.c,v 1.24.20.1 2023/07/31 15:23:02 martin Exp $ */
/******************************************************************************
* xenbus_comms.c
*
* Low level code to talks to Xen Store: ringbuffer and event channel.
*
* Copyright (C) 2005 Rusty Russell, IBM Corporation
- *
+ *
* This file may be distributed separately from the Linux kernel, or
* incorporated into other software packages, subject to the following license:
- *
+ *
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this source file (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use, copy, modify,
* merge, publish, distribute, sublicense, and/or sell copies of the Software,
* and to permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
- *
+ *
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
- *
+ *
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
@@ -29,11 +29,11 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: xenbus_comms.c,v 1.24 2020/05/13 13:19:38 jdolecek Exp $");
+__KERNEL_RCSID(0, "$NetBSD: xenbus_comms.c,v 1.24.20.1 2023/07/31 15:23:02 martin Exp $");
#include <sys/types.h>
-#include <sys/null.h>
-#include <sys/errno.h>
+#include <sys/null.h>
+#include <sys/errno.h>
#include <sys/param.h>
#include <sys/proc.h>
#include <sys/systm.h>
@@ -80,7 +80,7 @@ wake_waiting(void *arg)
{
if (__predict_false(xenstored_ready == 0 && xendomain_is_dom0())) {
xb_xenstored_make_ready();
- }
+ }
mutex_enter(&xenstore_lock);
cv_broadcast(&xenstore_cv);
@@ -153,9 +153,9 @@ xb_write(const void *data, unsigned len)
len -= avail;
/* Other side must not see new header until data is there. */
- xen_rmb();
+ xen_wmb();
intf->req_prod += avail;
- xen_rmb();
+ xen_wmb();
hypervisor_notify_via_evtchn(xen_start_info.store_evtchn);
}
@@ -202,9 +202,9 @@ xb_read(void *data, unsigned len)
len -= avail;
/* Other side must not see free space until we've copied out */
- xen_rmb();
+ xen_wmb();
intf->rsp_cons += avail;
- xen_rmb();
+ xen_wmb();
XENPRINTF(("Finished read of %i bytes (%i to go)\n",
avail, len));
