Re: svn commit: r356194 - head/sys/dev/random
On 12/30/19, Conrad Meyer wrote: > Author: cem > Date: Mon Dec 30 01:38:19 2019 > New Revision: 356194 > URL: https://svnweb.freebsd.org/changeset/base/356194 > > Log: > random(4): Make entropy source deregistration safe > > Allow loadable modules that provide random entropy source(s) to safely > unload. Prior to this change, no driver could ensure that their > random_source structure was not being used by random_harvestq.c for any > period of time after invoking random_source_deregister(). > > This change converts the source_list LIST to a ConcurrencyKit CK_LIST and > uses an epoch(9) to protect typical read accesses of the list. The > existing > HARVEST_LOCK spin mutex is used to safely add and remove list entries. > random_source_deregister() uses epoch_wait() to ensure no concurrent > source_list readers are accessing a random_source before freeing the list > item and returning to the caller. > > Callers can safely unload immediately after random_source_deregister() > returns. > What's the point of using epoch for this one instead of rmlocks? rmlocks have the upshot of being faster single-threaded for the case you want (no fences). epoch would make sense if the list was expected to change over time, -- Mateusz Guzik ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r356054 - in head: share/mk stand
On 2019-Dec-24, at 21:26, Mark Millard wrote: > In: > > +# Defines a variable for Binutils linker, to be used to workaround some > +# issue with LLVM LLD (i.e. support for PowerPC32 bit on PowerPC64) > +# > +# This is an unavoidable cross coupling with Makefile.inc1 and > +# normal builds works when CROSS_BINUTILS_PREFIX and could be removed > +# when LLD PowerPC 32 bit support is completed > +.if defined(CROSS_BINUTILS_PREFIX) > +LD_BFD=${LOCALBASE}/bin/${CROSS_BINUTILS_PREFIX}-ld.bfd > +.else > +LD_BFD=${OBJTOP}/tmp/usr/bin/ld.bfd > +.endif > > I'm confused by the 2 lines: > > +.if defined(CROSS_BINUTILS_PREFIX) > +LD_BFD=${LOCALBASE}/bin/${CROSS_BINUTILS_PREFIX}-ld.bfd > > Looking in /usr/local/share/toolchains/*.mk I see > CROSS_BINUTILS_PREFIX as a full path in each case > ( based on devel/freebsd-gcc[69] in my context): > > # grep -r CROSS_BINU /usr/local/share/toolchains/ | more > /usr/local/share/toolchains/aarch64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/aarch64-unknown-freebsd13.0/bin/ > /usr/local/share/toolchains/amd64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/x86_64-unknown-freebsd13.0/bin/ > /usr/local/share/toolchains/powerpc64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-unknown-freebsd13.0/bin/ > /usr/local/share/toolchains/aarch64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/aarch64-unknown-freebsd13.0/bin/ > /usr/local/share/toolchains/amd64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/x86_64-unknown-freebsd13.0/bin/ > /usr/local/share/toolchains/powerpc64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-unknown-freebsd13.0/bin/ > /usr/local/share/toolchains/powerpc-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc-unknown-freebsd13.0/bin/ > > And when I look in /usr/srcv/Makefile.inc1 I see: > > /usr/src/Makefile.inc1:CROSS_BINUTILS_PREFIX=/usr/local/${TARGET_TRIPLE}/bin/ > > Having an overall path structure like: > > ${LOCALBASE}/bin/usr/local/powerpc64-unknown-freebsd13.0/bin/-ld.bfd > > seems strange (using powerpc64 as an example). > I got to the point of trying a build that happens to have tried to use this code. It failed, reporting: --- libc.so.7.full --- building shared library libc.so.7 cc: error: invalid linker name in argument '-fuse-ld=/usr/local/bin//usr/local/powerpc64-unknown-freebsd13.0/bin/-ld.bfd' *** [libc.so.7.full] Error code 1 So I switched the code to: .if defined(CROSS_BINUTILS_PREFIX) LD_BFD=${CROSS_BINUTILS_PREFIX}ld.bfd .else LD_BFD=${OBJTOP}/tmp/usr/bin/ld.bfd .endif With that in place, the build completed. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356200 - in head/sys: cam/ctl dev/md dev/nvdimm geom kern
Author: mav Date: Mon Dec 30 03:13:38 2019 New Revision: 356200 URL: https://svnweb.freebsd.org/changeset/base/356200 Log: Use atomic for start_count in devstat_start_transaction(). Combined with earlier nstart/nend removal it allows to remove several locks from request path of GEOM and few other places. It would be cool if we had more SMP-friendly statistics, but this helps too. Sponsored by: iXsystems, Inc. Modified: head/sys/cam/ctl/ctl_backend_block.c head/sys/dev/md/md.c head/sys/dev/nvdimm/nvdimm_spa.c head/sys/dev/nvdimm/nvdimm_var.h head/sys/geom/geom_disk.c head/sys/geom/geom_io.c head/sys/kern/subr_devstat.c Modified: head/sys/cam/ctl/ctl_backend_block.c == --- head/sys/cam/ctl/ctl_backend_block.cMon Dec 30 02:56:47 2019 (r356199) +++ head/sys/cam/ctl/ctl_backend_block.cMon Dec 30 03:13:38 2019 (r356200) @@ -580,9 +580,7 @@ ctl_be_block_flush_file(struct ctl_be_block_lun *be_lu DPRINTF("entered\n"); binuptime(>ds_t0); - mtx_lock(_lun->io_lock); devstat_start_transaction(beio->lun->disk_stats, >ds_t0); - mtx_unlock(_lun->io_lock); (void) vn_start_write(be_lun->vn, , V_WAIT); @@ -663,9 +661,7 @@ ctl_be_block_dispatch_file(struct ctl_be_block_lun *be } binuptime(>ds_t0); - mtx_lock(_lun->io_lock); devstat_start_transaction(beio->lun->disk_stats, >ds_t0); - mtx_unlock(_lun->io_lock); if (beio->bio_cmd == BIO_READ) { vn_lock(be_lun->vn, LK_SHARED | LK_RETRY); @@ -894,9 +890,7 @@ ctl_be_block_dispatch_zvol(struct ctl_be_block_lun *be } binuptime(>ds_t0); - mtx_lock(_lun->io_lock); devstat_start_transaction(beio->lun->disk_stats, >ds_t0); - mtx_unlock(_lun->io_lock); csw = devvn_refthread(be_lun->vn, , ); if (csw) { @@ -1034,9 +1028,7 @@ ctl_be_block_flush_dev(struct ctl_be_block_lun *be_lun beio->send_complete = 1; binuptime(>ds_t0); - mtx_lock(_lun->io_lock); devstat_start_transaction(be_lun->disk_stats, >ds_t0); - mtx_unlock(_lun->io_lock); csw = devvn_refthread(be_lun->vn, , ); if (csw) { @@ -1107,9 +1099,7 @@ ctl_be_block_unmap_dev(struct ctl_be_block_lun *be_lun DPRINTF("entered\n"); binuptime(>ds_t0); - mtx_lock(_lun->io_lock); devstat_start_transaction(be_lun->disk_stats, >ds_t0); - mtx_unlock(_lun->io_lock); if (beio->io_offset == -1) { beio->io_len = 0; @@ -1186,11 +1176,9 @@ ctl_be_block_dispatch_dev(struct ctl_be_block_lun *be_ beio->num_bios_sent++; } } + beio->send_complete = 1; binuptime(>ds_t0); - mtx_lock(_lun->io_lock); devstat_start_transaction(be_lun->disk_stats, >ds_t0); - beio->send_complete = 1; - mtx_unlock(_lun->io_lock); /* * Fire off all allocated requests! Modified: head/sys/dev/md/md.c == --- head/sys/dev/md/md.cMon Dec 30 02:56:47 2019(r356199) +++ head/sys/dev/md/md.cMon Dec 30 03:13:38 2019(r356200) @@ -247,7 +247,6 @@ struct md_s { LIST_ENTRY(md_s) list; struct bio_queue_head bio_queue; struct mtx queue_mtx; - struct mtx stat_mtx; struct cdev *dev; enum md_types type; off_t mediasize; @@ -477,9 +476,7 @@ g_md_start(struct bio *bp) sc = bp->bio_to->geom->softc; if ((bp->bio_cmd == BIO_READ) || (bp->bio_cmd == BIO_WRITE)) { - mtx_lock(>stat_mtx); devstat_start_transaction_bio(sc->devstat, bp); - mtx_unlock(>stat_mtx); } mtx_lock(>queue_mtx); bioq_disksort(>bio_queue, bp); @@ -1274,7 +1271,6 @@ mdnew(int unit, int *errp, enum md_types type) sc->type = type; bioq_init(>bio_queue); mtx_init(>queue_mtx, "md bio queue", NULL, MTX_DEF); - mtx_init(>stat_mtx, "md stat", NULL, MTX_DEF); sc->unit = unit; sprintf(sc->name, "md%d", unit); LIST_INSERT_HEAD(_softc_list, sc, list); @@ -1282,7 +1278,6 @@ mdnew(int unit, int *errp, enum md_types type) if (error == 0) return (sc); LIST_REMOVE(sc, list); - mtx_destroy(>stat_mtx); mtx_destroy(>queue_mtx); free_unr(md_uh, sc->unit); free(sc, M_MD); @@ -1512,7 +1507,6 @@ mddestroy(struct md_s *sc, struct thread *td) while (!(sc->flags & MD_EXITING)) msleep(sc->procp, >queue_mtx, PRIBIO, "mddestroy", hz / 10); mtx_unlock(>queue_mtx); - mtx_destroy(>stat_mtx); mtx_destroy(>queue_mtx); if (sc->vnode != NULL) { vn_lock(sc->vnode, LK_EXCLUSIVE | LK_RETRY); Modified:
svn commit: r356199 - in head/sys/powerpc: include powernv
Author: bdragon Date: Mon Dec 30 02:56:47 2019 New Revision: 356199 URL: https://svnweb.freebsd.org/changeset/base/356199 Log: [PowerPC] Fix panic when attempting to handle an HMI from an idle thread In IRC, sfs_ finally managed to get a good trace of a kernel panic that was happening when attempting to use webengine. As it turns out, we were using vtophys() from interrupt context on an idle thread in opal_hmi_handler2(). Since this involves locking the kernel pmap on PPC64 at the moment, this ended up tripping a KASSERT in mtx_lock(), which then caused a parallel panic stampede. So, avoid this by preallocating the flags variable and storing it in PCPU. Fixes "panic: mtx_lock() by idle thread 0x... on sleep mutex kernelpmap". Differential Revision:https://reviews.freebsd.org/D22962 Modified: head/sys/powerpc/include/pcpu.h head/sys/powerpc/powernv/opal_hmi.c Modified: head/sys/powerpc/include/pcpu.h == --- head/sys/powerpc/include/pcpu.h Mon Dec 30 02:40:55 2019 (r356198) +++ head/sys/powerpc/include/pcpu.h Mon Dec 30 02:56:47 2019 (r356199) @@ -68,7 +68,8 @@ struct pvo_entry; uint8_t slbstack[1024]; \ struct pvo_entry *qmap_pvo; \ struct mtx qmap_lock; \ - char__pad[1345]; + uint64_topal_hmi_flags; \ + char__pad[1337]; #ifdef __powerpc64__ #define PCPU_MD_AIM_FIELDS PCPU_MD_AIM64_FIELDS Modified: head/sys/powerpc/powernv/opal_hmi.c == --- head/sys/powerpc/powernv/opal_hmi.c Mon Dec 30 02:40:55 2019 (r356198) +++ head/sys/powerpc/powernv/opal_hmi.c Mon Dec 30 02:56:47 2019 (r356199) @@ -84,13 +84,17 @@ opal_hmi_event_handler(void *unused, struct opal_msg * static int opal_hmi_handler2(struct trapframe *frame) { - uint64_t flags; + /* +* Use DMAP preallocated pcpu memory to handle +* the phys flags pointer. +*/ + uint64_t *flags = PCPU_PTR(aim.opal_hmi_flags); int err; - flags = 0; - err = opal_call(OPAL_HANDLE_HMI2, vtophys()); + *flags = 0; + err = opal_call(OPAL_HANDLE_HMI2, DMAP_TO_PHYS((vm_offset_t)flags)); - if (flags & OPAL_HMI_FLAGS_TOD_TB_FAIL) + if (*flags & OPAL_HMI_FLAGS_TOD_TB_FAIL) panic("TOD/TB recovery failure"); if (err == OPAL_SUCCESS) ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356196 - in head/sys: arm/arm isa mips/mips powerpc/powerpc sparc64/sparc64
Author: kevans Date: Mon Dec 30 02:07:55 2019 New Revision: 356196 URL: https://svnweb.freebsd.org/changeset/base/356196 Log: sc(4) md bits: stop setting sc->kbd entirely The machdep parts no longer need to touch keyboard parts after r356043; sc->kbd will be 0-initialized and this works as expected. Modified: head/sys/arm/arm/sc_machdep.c head/sys/isa/syscons_isa.c head/sys/mips/mips/sc_machdep.c head/sys/powerpc/powerpc/sc_machdep.c head/sys/sparc64/sparc64/sc_machdep.c Modified: head/sys/arm/arm/sc_machdep.c == --- head/sys/arm/arm/sc_machdep.c Mon Dec 30 01:40:59 2019 (r356195) +++ head/sys/arm/arm/sc_machdep.c Mon Dec 30 02:07:55 2019 (r356196) @@ -69,7 +69,6 @@ sc_get_softc(int unit, int flags) sc = _softcs[unit]; sc->unit = unit; if ((sc->flags & SC_INIT_DONE) == 0) { - sc->kbd = NULL; sc->adapter = -1; sc->cursor_char = SC_CURSOR_CHAR; sc->mouse_char = SC_MOUSE_CHAR; Modified: head/sys/isa/syscons_isa.c == --- head/sys/isa/syscons_isa.c Mon Dec 30 01:40:59 2019(r356195) +++ head/sys/isa/syscons_isa.c Mon Dec 30 02:07:55 2019(r356196) @@ -117,7 +117,6 @@ sc_softc_t } sc->unit = unit; if ((sc->flags & SC_INIT_DONE) == 0) { - sc->kbd = NULL; sc->adapter = -1; sc->cursor_char = SC_CURSOR_CHAR; sc->mouse_char = SC_MOUSE_CHAR; Modified: head/sys/mips/mips/sc_machdep.c == --- head/sys/mips/mips/sc_machdep.c Mon Dec 30 01:40:59 2019 (r356195) +++ head/sys/mips/mips/sc_machdep.c Mon Dec 30 02:07:55 2019 (r356196) @@ -69,7 +69,6 @@ sc_get_softc(int unit, int flags) sc = _softcs[unit]; sc->unit = unit; if ((sc->flags & SC_INIT_DONE) == 0) { - sc->kbd = NULL; sc->adapter = -1; sc->cursor_char = SC_CURSOR_CHAR; sc->mouse_char = SC_MOUSE_CHAR; Modified: head/sys/powerpc/powerpc/sc_machdep.c == --- head/sys/powerpc/powerpc/sc_machdep.c Mon Dec 30 01:40:59 2019 (r356195) +++ head/sys/powerpc/powerpc/sc_machdep.c Mon Dec 30 02:07:55 2019 (r356196) @@ -69,7 +69,6 @@ sc_get_softc(int unit, int flags) sc = _softcs[unit]; sc->unit = unit; if ((sc->flags & SC_INIT_DONE) == 0) { - sc->kbd = NULL; sc->adapter = -1; sc->cursor_char = SC_CURSOR_CHAR; sc->mouse_char = SC_MOUSE_CHAR; Modified: head/sys/sparc64/sparc64/sc_machdep.c == --- head/sys/sparc64/sparc64/sc_machdep.c Mon Dec 30 01:40:59 2019 (r356195) +++ head/sys/sparc64/sparc64/sc_machdep.c Mon Dec 30 02:07:55 2019 (r356196) @@ -131,7 +131,6 @@ sc_get_softc(int unit, int flags) sc = _softcs[unit]; sc->unit = unit; if ((sc->flags & SC_INIT_DONE) == 0) { - sc->kbd = NULL; sc->adapter = -1; sc->cursor_char = SC_CURSOR_CHAR; sc->mouse_char = SC_MOUSE_CHAR; ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356195 - head/sys/dev/virtio/random
Author: cem Date: Mon Dec 30 01:40:59 2019 New Revision: 356195 URL: https://svnweb.freebsd.org/changeset/base/356195 Log: virtio_random(4): Drop no longer needed unload kludge After r356194, source deregistration no longer races with concurrent use. Reviewed by: bryanv, markm Differential Revision:https://reviews.freebsd.org/D22490 Modified: head/sys/dev/virtio/random/virtio_random.c Modified: head/sys/dev/virtio/random/virtio_random.c == --- head/sys/dev/virtio/random/virtio_random.c Mon Dec 30 01:38:19 2019 (r356194) +++ head/sys/dev/virtio/random/virtio_random.c Mon Dec 30 01:40:59 2019 (r356195) @@ -176,14 +176,6 @@ vtrnd_detach(device_t dev) random_source_deregister(_vtrnd); atomic_store_explicit(_vtrnd_softc, NULL, memory_order_release); - - /* -* Unfortunately, deregister does not guarantee our source callback -* will not be invoked after it returns. Use a kludge to prevent some, -* but not all, possible races. -*/ - tsleep_sbt(_vtrnd_softc, 0, "vtrnddet", mstosbt(50), 0, C_HARDCLOCK); - return (0); } ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356194 - head/sys/dev/random
Author: cem Date: Mon Dec 30 01:38:19 2019 New Revision: 356194 URL: https://svnweb.freebsd.org/changeset/base/356194 Log: random(4): Make entropy source deregistration safe Allow loadable modules that provide random entropy source(s) to safely unload. Prior to this change, no driver could ensure that their random_source structure was not being used by random_harvestq.c for any period of time after invoking random_source_deregister(). This change converts the source_list LIST to a ConcurrencyKit CK_LIST and uses an epoch(9) to protect typical read accesses of the list. The existing HARVEST_LOCK spin mutex is used to safely add and remove list entries. random_source_deregister() uses epoch_wait() to ensure no concurrent source_list readers are accessing a random_source before freeing the list item and returning to the caller. Callers can safely unload immediately after random_source_deregister() returns. Reviewed by: markj Approved by: csprng(markm) Discussed with: jhb Differential Revision:https://reviews.freebsd.org/D22489 Modified: head/sys/dev/random/random_harvestq.c Modified: head/sys/dev/random/random_harvestq.c == --- head/sys/dev/random/random_harvestq.c Mon Dec 30 00:49:49 2019 (r356193) +++ head/sys/dev/random/random_harvestq.c Mon Dec 30 01:38:19 2019 (r356194) @@ -34,7 +34,9 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include +#include #include #include #include @@ -76,6 +78,14 @@ static void random_sources_feed(void); static u_int read_rate; /* + * Random must initialize much earlier than epoch, but we can initialize the + * epoch code before SMP starts. Prior to SMP, we can safely bypass + * concurrency primitives. + */ +static __read_mostly bool epoch_inited; +static __read_mostly epoch_t rs_epoch; + +/* * How many events to queue up. We create this many items in * an 'empty' queue, then transfer them to the 'harvest' queue with * supplied junk. When used, they are transferred back to the @@ -94,12 +104,12 @@ volatile int random_kthread_control; __read_frequently u_int hc_source_mask; struct random_sources { - LIST_ENTRY(random_sources) rrs_entries; + CK_LIST_ENTRY(random_sources)rrs_entries; struct random_source*rrs_source; }; -static LIST_HEAD(sources_head, random_sources) source_list = -LIST_HEAD_INITIALIZER(source_list); +static CK_LIST_HEAD(sources_head, random_sources) source_list = +CK_LIST_HEAD_INITIALIZER(source_list); SYSCTL_NODE(_kern_random, OID_AUTO, harvest, CTLFLAG_RW, 0, "Entropy Device Parameters"); @@ -202,6 +212,14 @@ random_kthread(void) SYSINIT(random_device_h_proc, SI_SUB_KICK_SCHEDULER, SI_ORDER_ANY, kproc_start, _proc_kp); +static void +rs_epoch_init(void *dummy __unused) +{ + rs_epoch = epoch_alloc("Random Sources", EPOCH_PREEMPT); + epoch_inited = true; +} +SYSINIT(rs_epoch_init, SI_SUB_EPOCH, SI_ORDER_ANY, rs_epoch_init, NULL); + /* * Run through all fast sources reading entropy for the given * number of rounds, which should be a multiple of the number @@ -211,9 +229,13 @@ static void random_sources_feed(void) { uint32_t entropy[HARVESTSIZE]; + struct epoch_tracker et; struct random_sources *rrs; u_int i, n, local_read_rate; + bool rse_warm; + rse_warm = epoch_inited; + /* * Step over all of live entropy sources, and feed their output * to the system-wide RNG. @@ -223,7 +245,9 @@ random_sources_feed(void) local_read_rate = MAX(local_read_rate, 1); /* But not exceeding RANDOM_KEYSIZE_WORDS */ local_read_rate = MIN(local_read_rate, RANDOM_KEYSIZE_WORDS); - LIST_FOREACH(rrs, _list, rrs_entries) { + if (rse_warm) + epoch_enter_preempt(rs_epoch, ); + CK_LIST_FOREACH(rrs, _list, rrs_entries) { for (i = 0; i < p_random_alg_context->ra_poolcount*local_read_rate; i++) { n = rrs->rrs_source->rs_read(entropy, sizeof(entropy)); KASSERT((n <= sizeof(entropy)), ("%s: rs_read returned too much data (%u > %zu)", __func__, n, sizeof(entropy))); @@ -243,6 +267,8 @@ random_sources_feed(void) random_harvest_direct(entropy, n, rrs->rrs_source->rs_source); } } + if (rse_warm) + epoch_exit_preempt(rs_epoch, ); explicit_bzero(entropy, sizeof(entropy)); } @@ -573,7 +599,10 @@ random_source_register(struct random_source *rsource) random_harvest_register_source(rsource->rs_source); printf("random: registering fast source %s\n", rsource->rs_ident); - LIST_INSERT_HEAD(_list, rrs, rrs_entries); + + RANDOM_HARVEST_LOCK(); + CK_LIST_INSERT_HEAD(_list, rrs, rrs_entries); +
svn commit: r356192 - head/sys/geom
Author: mav Date: Mon Dec 30 00:46:10 2019 New Revision: 356192 URL: https://svnweb.freebsd.org/changeset/base/356192 Log: Retire nstart/nend counters. Those counters were abused for decade to workaround broken orphanization process in different classes by delaying the call while there are active requests. But from one side it did not close all the races, while from another was quite expensive on SMP due to trashing twice per request cache lines of consumer and provider and requiring locks. It lost its sense after I manually went through all the GEOM classes in base and made orphanization wait for either provider close or request completion. Consumer counters are still used under INVARIANTS to detect premature consumer close and detach. Provider counters are removed completely. Sponsored by: iXsystems, Inc. Modified: head/sys/geom/geom.h head/sys/geom/geom_event.c head/sys/geom/geom_io.c head/sys/geom/geom_subr.c Modified: head/sys/geom/geom.h == --- head/sys/geom/geom.hMon Dec 30 00:16:58 2019(r356191) +++ head/sys/geom/geom.hMon Dec 30 00:46:10 2019(r356192) @@ -218,7 +218,8 @@ struct g_provider { off_t stripesize; off_t stripeoffset; struct devstat *stat; - u_int nstart, nend; + u_int spare1; + u_int spare2; u_int flags; #define G_PF_WITHER0x2 #define G_PF_ORPHAN0x4 Modified: head/sys/geom/geom_event.c == --- head/sys/geom/geom_event.c Mon Dec 30 00:16:58 2019(r356191) +++ head/sys/geom/geom_event.c Mon Dec 30 00:46:10 2019(r356192) @@ -241,10 +241,7 @@ one_event(void) g_topology_assert(); mtx_lock(_eventlock); - TAILQ_FOREACH(pp, _doorstep, orphan) { - if (pp->nstart == pp->nend) - break; - } + pp = TAILQ_FIRST(_doorstep); if (pp != NULL) { G_VALID_PROVIDER(pp); TAILQ_REMOVE(_doorstep, pp, orphan); @@ -299,7 +296,7 @@ g_run_events() } else { g_topology_unlock(); msleep(_wait_event, _eventlock, PRIBIO | PDROP, - "-", TAILQ_EMPTY(_doorstep) ? 0 : hz / 10); + "-", 0); } } /* NOTREACHED */ Modified: head/sys/geom/geom_io.c == --- head/sys/geom/geom_io.c Mon Dec 30 00:16:58 2019(r356191) +++ head/sys/geom/geom_io.c Mon Dec 30 00:46:10 2019(r356192) @@ -580,8 +580,9 @@ g_io_request(struct bio *bp, struct g_consumer *cp) devstat_start_transaction(pp->stat, >bio_t0); if (g_collectstats & G_STATS_CONSUMERS) devstat_start_transaction(cp->stat, >bio_t0); - pp->nstart++; +#ifdef INVARIANTS cp->nstart++; +#endif mtx_unlock(mtxp); if (direct) { @@ -691,8 +692,9 @@ g_io_deliver(struct bio *bp, int error) devstat_end_transaction_bio_bt(pp->stat, bp, ); if (g_collectstats & G_STATS_CONSUMERS) devstat_end_transaction_bio_bt(cp->stat, bp, ); +#ifdef INVARIANTS cp->nend++; - pp->nend++; +#endif mtx_unlock(mtxp); if (error != ENOMEM) { Modified: head/sys/geom/geom_subr.c == --- head/sys/geom/geom_subr.c Mon Dec 30 00:16:58 2019(r356191) +++ head/sys/geom/geom_subr.c Mon Dec 30 00:46:10 2019(r356192) @@ -939,6 +939,9 @@ g_access(struct g_consumer *cp, int dcr, int dcw, int KASSERT(cp->acw + dcw >= 0, ("access resulting in negative acw")); KASSERT(cp->ace + dce >= 0, ("access resulting in negative ace")); KASSERT(dcr != 0 || dcw != 0 || dce != 0, ("NOP access request")); + KASSERT(cp->acr + dcr != 0 || cp->acw + dcw != 0 || + cp->ace + dce != 0 || cp->nstart == cp->nend, + ("Last close with active requests")); KASSERT(gp->access != NULL, ("NULL geom->access")); /* @@ -1426,8 +1429,10 @@ db_show_geom_consumer(int indent, struct g_consumer *c } gprintln(" access: r%dw%de%d", cp->acr, cp->acw, cp->ace); gprintln(" flags:0x%04x", cp->flags); +#ifdef INVARIANTS gprintln(" nstart: %u", cp->nstart); gprintln(" nend: %u", cp->nend); +#endif } else { gprintf("consumer: %p (%s), access=r%dw%de%d", cp, cp->provider != NULL ? cp->provider->name : "none", @@ -1459,8 +1464,6 @@ db_show_geom_provider(int
Re: svn commit: r356159 - head/sys/vm
Thanks for the detailed answer Mark! On Sunday, December 29, 2019, Mark Johnston wrote: > On Sun, Dec 29, 2019 at 03:39:55AM +0100, Oliver Pinter wrote: > > Is there any performance measurement from before and after. It would be > > nice to see them. > > I did not do extensive benchmarking. The aim of the patch set was > simply to remove the use of the hashed page lock, since it shows up > prominently in lock profiles of some workloads. The problem is that we > acquire these locks any time a page's LRU state is updated, and the use > of the hash lock means that we get false sharing. The solution is to > implement these state updates using atomic operations on the page > structure itself, making data contention much less likely. Another > option was to embed a mutex into the vm_page structure, but this would > bloat a structure which is already too large. > > A secondary goal was to reduce the number of locks held during page > queue scans. Such scans frequently call pmap_ts_referenced() to collect > info about recent references to the page. This operation can be > expensive since it may require a TLB shootdown, and it can block for a > long time on the pmap lock, for example if the lock holder is copying > the page tables as part of a fork(). Now, the active queue scan body is > executed without any locks held, so a page daemon thread blocked on a > pmap lock no longer has the potential to block other threads by holding > on to a shared page lock. Before, the page daemon could block faulting > threads for a long time, hurting latency. I don't have any benchmarks > that capture this, but it's something that I've observed in production > workloads. > > I used some microbenchmarks to verify that the change did not penalize > the single-threaded case. Here are some results on a 64-core arm64 > system I have been playing with: > https://people.freebsd.org/~markj/arm64_page_lock/ > > The benchmark from will-it-scale simply maps 128MB of anonymous memory, > faults on each page, and unmaps it, in a loop. In the fault handler we > allocate a page and insert it into the active queue, and the unmap > operation removes all of those pages from the queue. I collected the > throughput for 1, 2, 4, 8, 16 and 32 concurrent processes. > > With my patches we see some modest gains at low concurrency. At higher > levels of concurrency we actually get lower throughput than before as > contention moves from the page locks and the page queue lock to just the > page queue lock. I don't believe this is a real regression: first, the > benchmark is quite extreme relative to any useful workload, and second, > arm64 suffers from using a much smaller batch size than amd64 for > batched page queue operations. Changing that pushes the results out > somewhat. Some earlier testing on a 2-socket Xeon system showed a > similar pattern with smaller differences. > ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356187 - head/cddl/contrib/opensolaris/lib/libdtrace/common
Author: markj Date: Sun Dec 29 21:46:50 2019 New Revision: 356187 URL: https://svnweb.freebsd.org/changeset/base/356187 Log: Add libdtrace support for arm64 USDT probes. arm64 is still lacking a fasttrap implementation, which is required to actually enable userland probes, but this at least allows USDT probes to be linked into userland applications. Submitted by: Klaus Küchemann (original) MFC after:2 weeks Differential Revision:https://reviews.freebsd.org/D22360 Modified: head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c Modified: head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c == --- head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.cSun Dec 29 21:46:00 2019(r356186) +++ head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.cSun Dec 29 21:46:50 2019(r356187) @@ -229,9 +229,10 @@ prepare_elf32(dtrace_hdl_t *dtp, const dof_hdr_t *dof, for (j = 0; j < nrel; j++) { #if defined(__aarch64__) -/* XXX */ - printf("%s:%s(%d): aarch64 not implemented\n", - __FUNCTION__, __FILE__, __LINE__); + rel->r_offset = s->dofs_offset + + dofr[j].dofr_offset; + rel->r_info = ELF32_R_INFO(count + dep->de_global, + R_ARM_REL32); #elif defined(__arm__) /* XXX */ printf("%s:%s(%d): arm not implemented\n", @@ -425,7 +426,10 @@ prepare_elf64(dtrace_hdl_t *dtp, const dof_hdr_t *dof, for (j = 0; j < nrel; j++) { #if defined(__aarch64__) -/* XXX */ + rel->r_offset = s->dofs_offset + + dofr[j].dofr_offset; + rel->r_info = ELF64_R_INFO(count + dep->de_global, + R_AARCH64_PREL64); #elif defined(__arm__) /* XXX */ #elif defined(__mips__) @@ -541,6 +545,8 @@ dump_elf32(dtrace_hdl_t *dtp, const dof_hdr_t *dof, in elf_file.ehdr.e_machine = EM_SPARC; #elif defined(__i386) || defined(__amd64) elf_file.ehdr.e_machine = EM_386; +#elif defined(__aarch64__) + elf_file.ehdr.e_machine = EM_AARCH64; #endif elf_file.ehdr.e_version = EV_CURRENT; elf_file.ehdr.e_shoff = sizeof (Elf32_Ehdr); @@ -687,6 +693,8 @@ dump_elf64(dtrace_hdl_t *dtp, const dof_hdr_t *dof, in elf_file.ehdr.e_machine = EM_SPARCV9; #elif defined(__i386) || defined(__amd64) elf_file.ehdr.e_machine = EM_AMD64; +#elif defined(__aarch64__) + elf_file.ehdr.e_machine = EM_AARCH64; #endif elf_file.ehdr.e_version = EV_CURRENT; elf_file.ehdr.e_shoff = sizeof (Elf64_Ehdr); @@ -802,14 +810,66 @@ dt_symtab_lookup(Elf_Data *data_sym, int start, int en } #if defined(__aarch64__) -/* XXX */ +#defineDT_OP_NOP 0xd503201f +#defineDT_OP_RET 0xd65f03c0 +#defineDT_OP_CALL260x9400 +#defineDT_OP_JUMP260x1400 + static int dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, GElf_Rela *rela, uint32_t *off) { - printf("%s:%s(%d): aarch64 not implemented\n", __FUNCTION__, __FILE__, - __LINE__); - return (-1); + uint32_t *ip; + + /* +* Ensure that the offset is aligned on an instruction boundary. +*/ + if ((rela->r_offset & (sizeof (uint32_t) - 1)) != 0) + return (-1); + + /* +* We only know about some specific relocation types. +* We also recognize relocation type NONE, since that gets used for +* relocations of USDT probes, and we might be re-processing a file. +*/ + if (GELF_R_TYPE(rela->r_info) != R_AARCH64_CALL26 && + GELF_R_TYPE(rela->r_info) != R_AARCH64_JUMP26 && + GELF_R_TYPE(rela->r_info) != R_AARCH64_NONE) + return (-1); + + ip = (uint32_t *)(p + rela->r_offset); + + /* +* We may have already processed this object file in an earlier linker +* invocation. Check to see if the present instruction sequence matches +* the one we would install below. +*/ + if (ip[0] == DT_OP_NOP || ip[0] == DT_OP_RET) + return (0); + + /* +* We only expect call instructions with a displacement of 0, or a jump +* instruction acting as a tail call. +*/ + if (ip[0] != DT_OP_CALL26 && ip[0] != DT_OP_JUMP26) { + dt_dprintf("found %x instead of a call or jmp instruction at " + "%llx\n", ip[0], (u_longlong_t)rela->r_offset); + return (-1); + } + + /* +* On arm64, we do not have to differentiate between regular probes and +* is-enabled probes. Both cases are encoded as a regular branch for +* non-tail call locations, and a jump for tail
svn commit: r356186 - head/sys/sys
Author: markj Date: Sun Dec 29 21:46:00 2019 New Revision: 356186 URL: https://svnweb.freebsd.org/changeset/base/356186 Log: Add ARMv8 static relocation types used for control flow instructions. MFC after:1 week Sponsored by: The FreeBSD Foundation Modified: head/sys/sys/elf_common.h Modified: head/sys/sys/elf_common.h == --- head/sys/sys/elf_common.h Sun Dec 29 21:16:03 2019(r356185) +++ head/sys/sys/elf_common.h Sun Dec 29 21:46:00 2019(r356186) @@ -999,6 +999,10 @@ typedef struct { #defineR_AARCH64_PREL64260 /* PC relative */ #defineR_AARCH64_PREL32261 /* PC relative, 32-bit overflow check */ #defineR_AARCH64_PREL16262 /* PC relative, 16-bit overflow check */ +#defineR_AARCH64_TSTBR14 279 /* TBZ/TBNZ immediate */ +#defineR_AARCH64_CONDBR19 280 /* Conditional branch immediate */ +#defineR_AARCH64_JUMP26282 /* Branch immediate */ +#defineR_AARCH64_CALL26283 /* Call immediate */ #defineR_AARCH64_COPY 1024/* Copy data from shared object */ #defineR_AARCH64_GLOB_DAT 1025/* Set GOT entry to data address */ #defineR_AARCH64_JUMP_SLOT 1026/* Set GOT entry to code address */ ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356185 - in head: lib/geom lib/geom/sched sys/geom sys/geom/sched sys/modules/geom sys/modules/geom/geom_sched sys/sys
Author: mav Date: Sun Dec 29 21:16:03 2019 New Revision: 356185 URL: https://svnweb.freebsd.org/changeset/base/356185 Log: Remove GEOM_SCHED class and gsched tool. This code was not actively maintained since it was introduced 10 years ago. It lacks support for many later GEOM features, such as direct dispatch, unmapped I/O, stripesize/stripeoffset, resize, etc. Plus it is the only remaining use of GEOM nstart/nend request counters, used there to implement live insertion/removal, questionable by itself. Plus, as number of people commented, GEOM is not the best place for I/O scheduler, since it has limited information about layers both above and below it, required for efficient scheduling. Plus with the modern shift to SSDs there is just no more significant need for this kind of scheduling. Approved by: imp, phk, luigi Relnotes: yes Deleted: head/lib/geom/sched/ head/sys/geom/sched/ head/sys/modules/geom/geom_sched/ Modified: head/lib/geom/Makefile.classes head/sys/geom/geom.h head/sys/geom/geom_io.c head/sys/modules/geom/Makefile head/sys/sys/bio.h head/sys/sys/ktr_class.h head/sys/sys/param.h Modified: head/lib/geom/Makefile.classes == --- head/lib/geom/Makefile.classes Sun Dec 29 20:57:49 2019 (r356184) +++ head/lib/geom/Makefile.classes Sun Dec 29 21:16:03 2019 (r356185) @@ -20,7 +20,6 @@ GEOM_CLASSES+=nop GEOM_CLASSES+= part GEOM_CLASSES+= raid GEOM_CLASSES+= raid3 -GEOM_CLASSES+= sched GEOM_CLASSES+= shsec GEOM_CLASSES+= stripe GEOM_CLASSES+= virstor Modified: head/sys/geom/geom.h == --- head/sys/geom/geom.hSun Dec 29 20:57:49 2019(r356184) +++ head/sys/geom/geom.hSun Dec 29 21:16:03 2019(r356185) @@ -231,17 +231,6 @@ struct g_provider { u_int index; }; -/* - * Descriptor of a classifier. We can register a function and - * an argument, which is called by g_io_request() on bio's - * that are not previously classified. - */ -struct g_classifier_hook { - TAILQ_ENTRY(g_classifier_hook) link; - int (*func)(void *arg, struct bio *bp); - void*arg; -}; - /* BIO_GETATTR("GEOM::setstate") argument values. */ #define G_STATE_FAILED 0 #define G_STATE_REBUILD1 @@ -344,8 +333,6 @@ int g_io_getattr(const char *attr, struct g_consumer * int g_io_zonecmd(struct disk_zone_args *zone_args, struct g_consumer *cp); int g_io_flush(struct g_consumer *cp); int g_io_speedup(size_t shortage, u_int flags, size_t *resid, struct g_consumer *cp); -int g_register_classifier(struct g_classifier_hook *hook); -void g_unregister_classifier(struct g_classifier_hook *hook); void g_io_request(struct bio *bp, struct g_consumer *cp); struct bio *g_new_bio(void); struct bio *g_alloc_bio(void); Modified: head/sys/geom/geom_io.c == --- head/sys/geom/geom_io.c Sun Dec 29 20:57:49 2019(r356184) +++ head/sys/geom/geom_io.c Sun Dec 29 21:16:03 2019(r356185) @@ -87,15 +87,6 @@ static volatile u_int __read_mostly pace; static uma_zone_t __read_mostly biozone; -/* - * The head of the list of classifiers used in g_io_request. - * Use g_register_classifier() and g_unregister_classifier() - * to add/remove entries to the list. - * Classifiers are invoked in registration order. - */ -static TAILQ_HEAD(, g_classifier_hook) g_classifier_tailq __read_mostly = -TAILQ_HEAD_INITIALIZER(g_classifier_tailq); - #include static void @@ -224,9 +215,6 @@ g_clone_bio(struct bio *bp) if (bp->bio_cmd == BIO_ZONE) bcopy(>bio_zone, >bio_zone, sizeof(bp->bio_zone)); - /* Inherit classification info from the parent */ - bp2->bio_classifier1 = bp->bio_classifier1; - bp2->bio_classifier2 = bp->bio_classifier2; #if defined(BUF_TRACKING) || defined(FULL_BUF_TRACKING) bp2->bio_track_bp = bp->bio_track_bp; #endif @@ -498,66 +486,7 @@ g_io_check(struct bio *bp) return (EJUSTRETURN); } -/* - * bio classification support. - * - * g_register_classifier() and g_unregister_classifier() - * are used to add/remove a classifier from the list. - * The list is protected using the g_bio_run_down lock, - * because the classifiers are called in this path. - * - * g_io_request() passes bio's that are not already classified - * (i.e. those with bio_classifier1 == NULL) to g_run_classifiers(). - * Classifiers can store their result in the two fields - * bio_classifier1 and bio_classifier2. - * A classifier that updates one of the fields should - * return a non-zero value. - * If no classifier updates the field, g_run_classifiers() sets - *
svn commit: r356184 - head/usr.sbin/powerd
Author: markj Date: Sun Dec 29 20:57:49 2019 New Revision: 356184 URL: https://svnweb.freebsd.org/changeset/base/356184 Log: powerd(8): Fix a memory leak when we fail to read frequency levels. PR: 242967 Submitted by: Hans Christian Woithe MFC after:3 days Modified: head/usr.sbin/powerd/powerd.c Modified: head/usr.sbin/powerd/powerd.c == --- head/usr.sbin/powerd/powerd.c Sun Dec 29 20:01:03 2019 (r356183) +++ head/usr.sbin/powerd/powerd.c Sun Dec 29 20:57:49 2019 (r356184) @@ -196,8 +196,10 @@ read_freqs(int *numfreqs, int **freqs, int **power, in return (-1); if ((freqstr = malloc(len)) == NULL) return (-1); - if (sysctl(levels_mib, 4, freqstr, , NULL, 0)) + if (sysctl(levels_mib, 4, freqstr, , NULL, 0)) { + free(freqstr); return (-1); + } *numfreqs = 1; for (p = freqstr; *p != '\0'; p++) ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356183 - head/sys/vm
Author: markj Date: Sun Dec 29 20:01:03 2019 New Revision: 356183 URL: https://svnweb.freebsd.org/changeset/base/356183 Log: Restore a vm_page_wired() check in vm_page_mvqueue() after r356156. We now set PGA_DEQUEUE on a managed page when it is wired after allocation, and vm_page_mvqueue() ignores pages with this flag set, ensuring that they do not end up in the page queues. However, this is not sufficient for managed fictitious pages or pages managed by the TTM. In particular, the TTM makes use of the plinks.q queue linkage fields for its own purposes. PR: 242961 Reported and tested by: Greg V Modified: head/sys/vm/vm_page.c Modified: head/sys/vm/vm_page.c == --- head/sys/vm/vm_page.c Sun Dec 29 19:33:41 2019(r356182) +++ head/sys/vm/vm_page.c Sun Dec 29 20:01:03 2019(r356183) @@ -4010,7 +4010,7 @@ vm_page_mvqueue(vm_page_t m, const uint8_t nqueue, con KASSERT(nflag == PGA_REQUEUE || nflag == PGA_REQUEUE_HEAD, ("%s: invalid flags %x", __func__, nflag)); - if ((m->oflags & VPO_UNMANAGED) != 0) + if ((m->oflags & VPO_UNMANAGED) != 0 || vm_page_wired(m)) return; old = vm_page_astate_load(m); ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356182 - head/sys/geom/shsec
Author: mav Date: Sun Dec 29 19:33:41 2019 New Revision: 356182 URL: https://svnweb.freebsd.org/changeset/base/356182 Log: Missed part of r356162. If we postpone consumer destruction till close, then the close calls should not be ignored. Delay geom withering till the last close too. MFC after:2 weeks X-MFC-with: r356162 Sponsored by: iXsystems, Inc. Modified: head/sys/geom/shsec/g_shsec.c Modified: head/sys/geom/shsec/g_shsec.c == --- head/sys/geom/shsec/g_shsec.c Sun Dec 29 19:21:29 2019 (r356181) +++ head/sys/geom/shsec/g_shsec.c Sun Dec 29 19:33:41 2019 (r356182) @@ -184,7 +184,7 @@ g_shsec_orphan(struct g_consumer *cp) g_shsec_remove_disk(cp); /* If there are no valid disks anymore, remove device. */ - if (g_shsec_nvalid(sc) == 0) + if (LIST_EMPTY(>consumer)) g_shsec_destroy(sc, 1); } @@ -199,21 +199,6 @@ g_shsec_access(struct g_provider *pp, int dr, int dw, gp = pp->geom; sc = gp->softc; - if (sc == NULL) { - /* -* It looks like geom is being withered. -* In that case we allow only negative requests. -*/ - KASSERT(dr <= 0 && dw <= 0 && de <= 0, - ("Positive access request (device=%s).", pp->name)); - if ((pp->acr + dr) == 0 && (pp->acw + dw) == 0 && - (pp->ace + de) == 0) { - G_SHSEC_DEBUG(0, "Device %s definitely destroyed.", - gp->name); - } - return (0); - } - /* On first open, grab an extra "exclusive" bit */ if (pp->acr == 0 && pp->acw == 0 && pp->ace == 0) de++; @@ -232,6 +217,11 @@ g_shsec_access(struct g_provider *pp, int dr, int dw, g_destroy_consumer(cp1); } } + + /* If there are no valid disks anymore, remove device. */ + if (LIST_EMPTY(>consumer)) + g_shsec_destroy(sc, 1); + return (error); fail: ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356181 - head/sys/geom/virstor
Author: mav Date: Sun Dec 29 19:21:29 2019 New Revision: 356181 URL: https://svnweb.freebsd.org/changeset/base/356181 Log: Fix GEOM_VIRSTOR orphanization. Previous code closed and destroyed consumer even with I/O in progress. This patch postpones the destruction till the last close. MFC after:2 weeks Sponsored by: iXsystems, Inc. Modified: head/sys/geom/virstor/g_virstor.c Modified: head/sys/geom/virstor/g_virstor.c == --- head/sys/geom/virstor/g_virstor.c Sun Dec 29 18:17:12 2019 (r356180) +++ head/sys/geom/virstor/g_virstor.c Sun Dec 29 19:21:29 2019 (r356181) @@ -909,7 +909,7 @@ remove_component(struct g_virstor_softc *sc, struct g_ } if (c->acr > 0 || c->acw > 0 || c->ace > 0) - g_access(c, -c->acr, -c->acw, -c->ace); + return; if (delay) { /* Destroy consumer after it's tasted */ g_post_event(delay_destroy_consumer, c, M_WAITOK, NULL); @@ -1400,7 +1400,7 @@ g_virstor_orphan(struct g_consumer *cp) KASSERT(comp != NULL, ("%s: No component in private part of consumer", __func__)); remove_component(sc, comp, FALSE); - if (virstor_valid_components(sc) == 0) + if (LIST_EMPTY(>consumer)) virstor_geom_destroy(sc, TRUE, FALSE); } @@ -1410,7 +1410,7 @@ g_virstor_orphan(struct g_consumer *cp) static int g_virstor_access(struct g_provider *pp, int dr, int dw, int de) { - struct g_consumer *c; + struct g_consumer *c, *c2, *tmp; struct g_virstor_softc *sc; struct g_geom *gp; int error; @@ -1420,46 +1420,40 @@ g_virstor_access(struct g_provider *pp, int dr, int dw KASSERT(gp != NULL, ("%s: NULL geom", __func__)); sc = gp->softc; - if (sc == NULL) { - /* It seems that .access can be called with negative dr,dw,dx -* in this case but I want to check for myself */ - LOG_MSG(LVL_WARNING, "access(%d, %d, %d) for %s", - dr, dw, de, pp->name); - /* This should only happen when geom is withered so -* allow only negative requests */ - KASSERT(dr <= 0 && dw <= 0 && de <= 0, - ("%s: Positive access for %s", __func__, pp->name)); - if (pp->acr + dr == 0 && pp->acw + dw == 0 && pp->ace + de == 0) - LOG_MSG(LVL_DEBUG, "Device %s definitely destroyed", - pp->name); - return (0); - } - /* Grab an exclusive bit to propagate on our consumers on first open */ if (pp->acr == 0 && pp->acw == 0 && pp->ace == 0) de++; /* ... drop it on close */ if (pp->acr + dr == 0 && pp->acw + dw == 0 && pp->ace + de == 0) { de--; - update_metadata(sc);/* Writes statistical information */ + if (sc != NULL) + update_metadata(sc); } error = ENXIO; - LIST_FOREACH(c, >consumer, consumer) { - KASSERT(c != NULL, ("%s: consumer is NULL", __func__)); + LIST_FOREACH_SAFE(c, >consumer, consumer, tmp) { error = g_access(c, dr, dw, de); - if (error != 0) { - struct g_consumer *c2; - - /* Backout earlier changes */ - LIST_FOREACH(c2, >consumer, consumer) { - if (c2 == c) /* all eariler components fixed */ - return (error); - g_access(c2, -dr, -dw, -de); - } + if (error != 0) + goto fail; + if (c->acr == 0 && c->acw == 0 && c->ace == 0 && + c->flags & G_CF_ORPHAN) { + g_detach(c); + g_destroy_consumer(c); } } + if (sc != NULL && LIST_EMPTY(>consumer)) + virstor_geom_destroy(sc, TRUE, FALSE); + + return (error); + +fail: + /* Backout earlier changes */ + LIST_FOREACH(c2, >consumer, consumer) { + if (c2 == c) + break; + g_access(c2, -dr, -dw, -de); + } return (error); } ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r356180 - head/sys/conf
On Sun, 2019-12-29 at 18:17 +, Ian Lepore wrote: > Author: ian > Date: Sun Dec 29 18:17:12 2019 > New Revision: 356180 > URL: https://svnweb.freebsd.org/changeset/base/356180 > > Log: > Eliminate the generated ldscript for arm and arm64, and strip $a/$d marker > symbols from the linked kernel. > > The main thrust of this change is to generate a kernel that has the arm > "marker" symbols stripped. Marker symbols start with $a, $d, $t or $x, and > are emitted by the compiler to tell other toolchain components about the > locations of data embedded in the instruction stream (literal-pool > stuff). They are used for generating mixed-endian binaries (which we don't > support). The linked kernel has approximately 21,000 such symbols in it, > wasting space (500K in kernel.full, 190K in the final linked kernel), and > sometimes obscuring function names in stack tracebacks. > > This change also simplifies the way the kernel is linked. Instead of using > sed to generate two different ldscript files to generate both an elf kernel > and a binary (elf headers stripped) kernel, we now use a single ldscript > that refers to a "text_start" symbol, and we provide the value for that > symbol using --defsym on the linker command line. > Doh! Differential Revision:https://reviews.freebsd.org/D22922 -- Ian ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356180 - head/sys/conf
Author: ian Date: Sun Dec 29 18:17:12 2019 New Revision: 356180 URL: https://svnweb.freebsd.org/changeset/base/356180 Log: Eliminate the generated ldscript for arm and arm64, and strip $a/$d marker symbols from the linked kernel. The main thrust of this change is to generate a kernel that has the arm "marker" symbols stripped. Marker symbols start with $a, $d, $t or $x, and are emitted by the compiler to tell other toolchain components about the locations of data embedded in the instruction stream (literal-pool stuff). They are used for generating mixed-endian binaries (which we don't support). The linked kernel has approximately 21,000 such symbols in it, wasting space (500K in kernel.full, 190K in the final linked kernel), and sometimes obscuring function names in stack tracebacks. This change also simplifies the way the kernel is linked. Instead of using sed to generate two different ldscript files to generate both an elf kernel and a binary (elf headers stripped) kernel, we now use a single ldscript that refers to a "text_start" symbol, and we provide the value for that symbol using --defsym on the linker command line. Modified: head/sys/conf/Makefile.arm head/sys/conf/Makefile.arm64 head/sys/conf/ldscript.arm head/sys/conf/ldscript.arm64 Modified: head/sys/conf/Makefile.arm == --- head/sys/conf/Makefile.arm Sun Dec 29 17:19:57 2019(r356179) +++ head/sys/conf/Makefile.arm Sun Dec 29 18:17:12 2019(r356180) @@ -32,9 +32,6 @@ S=../../.. INCLUDES+= -I$S/contrib/libfdt -I$S/gnu/dts/include -SYSTEM_LD:= ${SYSTEM_LD:$S/conf/ldscript.$M=ldscript.$M} -SYSTEM_DEP:= ${SYSTEM_DEP:$S/conf/ldscript.$M=ldscript.$M} - .if !defined(DEBUG) && !defined(PROFLEVEL) STRIP_FLAGS = -S .endif @@ -58,20 +55,41 @@ CFLAGS += -mllvm -arm-enable-ehabi KERNVIRTADDR= 0xc000 .endif +# Use a custom SYSTEM_LD command to generate the elf kernel, so we can +# set the text segment start address, and also strip the "arm mapping +# symbols" which have names like $a.0 and $d.2; see the document +# "ELF for the ARM architecture" for more info on the mapping symbols. +SYSTEM_LD= \ + ${SYSTEM_LD_BASECMD} \ + --defsym='text_start=${KERNVIRTADDR} + SIZEOF_HEADERS' \ + -o ${.TARGET} ${SYSTEM_OBJS} vers.o; \ + $(OBJCOPY) \ + --wildcard \ + --strip-symbol='$$[adt]*' \ + ${.TARGET} + +# Generate the .bin (no elf headers) kernel as an extra build output. +# We must relink to generate the .bin kernel, because without headers the +# location of everything changes. We also strip the ARM marker symbols. +KERNEL_EXTRA+= ${KERNEL_KO}.bin +KERNEL_EXTRA_INSTALL+= ${KERNEL_KO}.bin + +${KERNEL_KO}.bin: ${SYSTEM_DEP} vers.o + @echo "linking ${.TARGET}" + @${SYSTEM_LD_BASECMD} \ + --defsym='text_start=${KERNVIRTADDR}' \ + -o ${.TARGET} ${SYSTEM_OBJS} vers.o + ${SIZE} ${.TARGET} + @${OBJCOPY} \ + --wildcard \ + --strip-symbol='$$[adt]*' \ + --output-target=binary \ + ${.TARGET} + @chmod 755 ${.TARGET} + # hack because genassym.c includes sys/bus.h which includes these. genassym.o: bus_if.h device_if.h -SYSTEM_LD_ = ${LD} -m ${LD_EMULATION} -Bdynamic -T ldscript.$M.noheader \ - ${_LDFLAGS} --no-warn-mismatch --warn-common --export-dynamic \ - --dynamic-linker /red/herring \ - -o ${FULLKERNEL}.noheader -X ${SYSTEM_OBJS} vers.o -SYSTEM_LD_TAIL +=;sed s/" + SIZEOF_HEADERS"// ldscript.$M \ - >ldscript.$M.noheader; \ - ${SYSTEM_LD_}; \ - ${OBJCOPY} -S -O binary ${FULLKERNEL}.noheader \ - ${KERNEL_KO}.bin; \ - rm ${FULLKERNEL}.noheader - %BEFORE_DEPEND %OBJS @@ -84,10 +102,7 @@ SYSTEM_LD_TAIL +=;sed s/" + SIZEOF_HEADERS"// ldscript %CLEAN -CLEAN+=ldscript.$M ${KERNEL_KO}.bin ldscript.$M.noheader - -ldscript.$M: $S/conf/ldscript.$M - sed s/KERNVIRTADDR/${KERNVIRTADDR}/g > ldscript.$M < $S/conf/ldscript.$M +CLEAN+=${KERNEL_KO}.bin %RULES Modified: head/sys/conf/Makefile.arm64 == --- head/sys/conf/Makefile.arm64Sun Dec 29 17:19:57 2019 (r356179) +++ head/sys/conf/Makefile.arm64Sun Dec 29 18:17:12 2019 (r356180) @@ -27,24 +27,42 @@ S= ../../.. INCLUDES+= -I$S/contrib/libfdt -#SYSTEM_LD:= ${SYSTEM_LD:$S/conf/ldscript.$M=ldscript.$M} -#SYSTEM_DEP:= ${SYSTEM_DEP:$S/conf/ldscript.$M=ldscript.$M} +# Use a custom SYSTEM_LD command to generate the elf kernel, so we can +# set the text segment start address, and also strip the "arm mapping +# symbols" which have names like $a.0 and $d.2; see the document +# "ELF for the ARM architecture" for more info on the mapping symbols. +SYSTEM_LD= \ + ${SYSTEM_LD_BASECMD} \ +
svn commit: r356179 - head/lib/libc/gen
Author: kaktus Date: Sun Dec 29 17:19:57 2019 New Revision: 356179 URL: https://svnweb.freebsd.org/changeset/base/356179 Log: sysctlbyname(2): Remove temporary compatibility layer Remove temporary compatibility layer introduced in r351729. More that 3 months should be enough for everybody who runs HEAD to upgrade to the new kernel already. Reviewed by: imp, mjg (mentor) Approved by: mjg (mentor) Differential Revision:https://reviews.freebsd.org/D22958 Modified: head/lib/libc/gen/sysctlbyname.c Modified: head/lib/libc/gen/sysctlbyname.c == --- head/lib/libc/gen/sysctlbyname.cSun Dec 29 17:10:21 2019 (r356178) +++ head/lib/libc/gen/sysctlbyname.cSun Dec 29 17:19:57 2019 (r356179) @@ -33,10 +33,6 @@ __FBSDID("$FreeBSD$"); #include #include -#include "libc_private.h" - -#defineSYSCTLBYNAME_OSREL 1300045 - extern int __sysctlbyname(const char *name, size_t namelen, void *oldp, size_t *oldlenp, const void *newp, size_t newlen); @@ -44,16 +40,9 @@ int sysctlbyname(const char *name, void *oldp, size_t *oldlenp, const void *newp, size_t newlen) { - int oid[CTL_MAXNAME]; size_t len; - if (__getosreldate() >= SYSCTLBYNAME_OSREL) { - len = strlen(name); - return (__sysctlbyname(name, len, oldp, oldlenp, newp, - newlen)); - } - len = nitems(oid); - if (sysctlnametomib(name, oid, ) == -1) - return (-1); - return (sysctl(oid, len, oldp, oldlenp, newp, newlen)); + len = strlen(name); + return (__sysctlbyname(name, len, oldp, oldlenp, newp, + newlen)); } ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356178 - head/sys/geom/mountver
Author: mav Date: Sun Dec 29 17:10:21 2019 New Revision: 356178 URL: https://svnweb.freebsd.org/changeset/base/356178 Log: Fix GEOM_MOUNTVER orphanization. Previous code closed and detached consumer even with I/O still in progress. This patch adds locking and request counting to postpone the close till the last of running requests completes. MFC after:2 weeks Sponsored by: iXsystems, Inc. Modified: head/sys/geom/mountver/g_mountver.c Modified: head/sys/geom/mountver/g_mountver.c == --- head/sys/geom/mountver/g_mountver.c Sun Dec 29 15:53:55 2019 (r356177) +++ head/sys/geom/mountver/g_mountver.c Sun Dec 29 17:10:21 2019 (r356178) @@ -85,14 +85,29 @@ struct g_class g_mountver_class = { }; static void +g_mountver_detach(void *arg, int flags __unused) +{ + struct g_consumer *cp = arg; + + g_topology_assert(); + if (cp->acr > 0 || cp->acw > 0 || cp->ace > 0) + g_access(cp, -cp->acr, -cp->acw, -cp->ace); + g_detach(cp); +} + +static void g_mountver_done(struct bio *bp) { + struct g_mountver_softc *sc; struct g_geom *gp; + struct g_consumer *cp; struct bio *pbp; + cp = bp->bio_from; + gp = cp->geom; if (bp->bio_error != ENXIO) { g_std_done(bp); - return; + goto done; } /* @@ -101,32 +116,45 @@ g_mountver_done(struct bio *bp) * gets called. To work around that, we have to queue requests * that failed with ENXIO, in order to send them later. */ - gp = bp->bio_from->geom; - pbp = bp->bio_parent; KASSERT(pbp->bio_to == LIST_FIRST(>provider), ("parent request was for someone else")); g_destroy_bio(bp); pbp->bio_inbed++; g_mountver_queue(pbp); + +done: + sc = gp->softc; + mtx_lock(>sc_mtx); + if (--cp->index == 0 && sc->sc_orphaned) + g_post_event(g_mountver_detach, cp, M_NOWAIT, NULL); + mtx_unlock(>sc_mtx); } +/* + * Send the BIO down. The function is called with sc_mtx held to cover + * the race with orphan, but drops it before external calls. + */ static void -g_mountver_send(struct bio *bp) +g_mountver_send(struct g_geom *gp, struct bio *bp) { - struct g_geom *gp; + struct g_mountver_softc *sc = gp->softc; + struct g_consumer *cp; struct bio *cbp; - gp = bp->bio_to->geom; - + mtx_assert(>sc_mtx, MA_OWNED); cbp = g_clone_bio(bp); if (cbp == NULL) { + mtx_unlock(>sc_mtx); g_io_deliver(bp, ENOMEM); return; } + cp = LIST_FIRST(>consumer); + cp->index++; + mtx_unlock(>sc_mtx); cbp->bio_done = g_mountver_done; - g_io_request(cbp, LIST_FIRST(>consumer)); + g_io_request(cbp, cp); } static void @@ -152,10 +180,12 @@ g_mountver_send_queued(struct g_geom *gp) sc = gp->softc; mtx_lock(>sc_mtx); - while ((bp = TAILQ_FIRST(>sc_queue)) != NULL) { + while ((bp = TAILQ_FIRST(>sc_queue)) != NULL && !sc->sc_orphaned) { TAILQ_REMOVE(>sc_queue, bp, bio_queue); G_MOUNTVER_LOGREQ(bp, "Sending queued request."); - g_mountver_send(bp); + /* sc_mtx is dropped inside */ + g_mountver_send(gp, bp); + mtx_lock(>sc_mtx); } mtx_unlock(>sc_mtx); } @@ -171,8 +201,10 @@ g_mountver_discard_queued(struct g_geom *gp) mtx_lock(>sc_mtx); while ((bp = TAILQ_FIRST(>sc_queue)) != NULL) { TAILQ_REMOVE(>sc_queue, bp, bio_queue); + mtx_unlock(>sc_mtx); G_MOUNTVER_LOGREQ(bp, "Discarding queued request."); g_io_deliver(bp, ENXIO); + mtx_lock(>sc_mtx); } mtx_unlock(>sc_mtx); } @@ -192,7 +224,9 @@ g_mountver_start(struct bio *bp) * orphaning didn't happen yet. In that case, queue all subsequent * requests in order to maintain ordering. */ + mtx_lock(>sc_mtx); if (sc->sc_orphaned || !TAILQ_EMPTY(>sc_queue)) { + mtx_unlock(>sc_mtx); if (sc->sc_shutting_down) { G_MOUNTVER_LOGREQ(bp, "Discarding request due to shutdown."); g_io_deliver(bp, ENXIO); @@ -204,7 +238,8 @@ g_mountver_start(struct bio *bp) g_mountver_send_queued(gp); } else { G_MOUNTVER_LOGREQ(bp, "Sending request."); - g_mountver_send(bp); + /* sc_mtx is dropped inside */ + g_mountver_send(gp, bp); } } @@ -466,14 +501,17 @@ static void g_mountver_orphan(struct g_consumer *cp) { struct g_mountver_softc *sc; + int done; g_topology_assert(); sc = cp->geom->softc; +
Re: svn commit: r356159 - head/sys/vm
On Sun, Dec 29, 2019 at 03:39:55AM +0100, Oliver Pinter wrote: > Is there any performance measurement from before and after. It would be > nice to see them. I did not do extensive benchmarking. The aim of the patch set was simply to remove the use of the hashed page lock, since it shows up prominently in lock profiles of some workloads. The problem is that we acquire these locks any time a page's LRU state is updated, and the use of the hash lock means that we get false sharing. The solution is to implement these state updates using atomic operations on the page structure itself, making data contention much less likely. Another option was to embed a mutex into the vm_page structure, but this would bloat a structure which is already too large. A secondary goal was to reduce the number of locks held during page queue scans. Such scans frequently call pmap_ts_referenced() to collect info about recent references to the page. This operation can be expensive since it may require a TLB shootdown, and it can block for a long time on the pmap lock, for example if the lock holder is copying the page tables as part of a fork(). Now, the active queue scan body is executed without any locks held, so a page daemon thread blocked on a pmap lock no longer has the potential to block other threads by holding on to a shared page lock. Before, the page daemon could block faulting threads for a long time, hurting latency. I don't have any benchmarks that capture this, but it's something that I've observed in production workloads. I used some microbenchmarks to verify that the change did not penalize the single-threaded case. Here are some results on a 64-core arm64 system I have been playing with: https://people.freebsd.org/~markj/arm64_page_lock/ The benchmark from will-it-scale simply maps 128MB of anonymous memory, faults on each page, and unmaps it, in a loop. In the fault handler we allocate a page and insert it into the active queue, and the unmap operation removes all of those pages from the queue. I collected the throughput for 1, 2, 4, 8, 16 and 32 concurrent processes. With my patches we see some modest gains at low concurrency. At higher levels of concurrency we actually get lower throughput than before as contention moves from the page locks and the page queue lock to just the page queue lock. I don't believe this is a real regression: first, the benchmark is quite extreme relative to any useful workload, and second, arm64 suffers from using a much smaller batch size than amd64 for batched page queue operations. Changing that pushes the results out somewhat. Some earlier testing on a 2-socket Xeon system showed a similar pattern with smaller differences. ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356177 - in head/sys: amd64/linux amd64/linux32 arm64/linux compat/linux i386/linux
Author: trasz Date: Sun Dec 29 15:53:55 2019 New Revision: 356177 URL: https://svnweb.freebsd.org/changeset/base/356177 Log: Implement Linux syslog(2) syscall; just enough to make Linux dmesg(8) utility work. MFC after:2 weeks Sponsored by: The FreeBSD Foundation Differential Revision:https://reviews.freebsd.org/D22465 Modified: head/sys/amd64/linux/linux_dummy.c head/sys/amd64/linux32/linux32_dummy.c head/sys/arm64/linux/linux_dummy.c head/sys/compat/linux/linux_misc.c head/sys/compat/linux/linux_misc.h head/sys/i386/linux/linux_dummy.c Modified: head/sys/amd64/linux/linux_dummy.c == --- head/sys/amd64/linux/linux_dummy.c Sun Dec 29 15:47:37 2019 (r356176) +++ head/sys/amd64/linux/linux_dummy.c Sun Dec 29 15:53:55 2019 (r356177) @@ -60,7 +60,6 @@ UNIMPLEMENTED(uselib); UNIMPLEMENTED(vserver); DUMMY(sendfile); -DUMMY(syslog); DUMMY(setfsuid); DUMMY(setfsgid); DUMMY(sysfs); Modified: head/sys/amd64/linux32/linux32_dummy.c == --- head/sys/amd64/linux32/linux32_dummy.c Sun Dec 29 15:47:37 2019 (r356176) +++ head/sys/amd64/linux32/linux32_dummy.c Sun Dec 29 15:53:55 2019 (r356177) @@ -63,7 +63,6 @@ UNIMPLEMENTED(vserver); DUMMY(stime); DUMMY(olduname); -DUMMY(syslog); DUMMY(uname); DUMMY(vhangup); DUMMY(swapoff); Modified: head/sys/arm64/linux/linux_dummy.c == --- head/sys/arm64/linux/linux_dummy.c Sun Dec 29 15:47:37 2019 (r356176) +++ head/sys/arm64/linux/linux_dummy.c Sun Dec 29 15:53:55 2019 (r356177) @@ -65,7 +65,6 @@ UNIMPLEMENTED(uselib); UNIMPLEMENTED(vserver); DUMMY(sendfile); -DUMMY(syslog); DUMMY(setfsuid); DUMMY(setfsgid); DUMMY(vhangup); Modified: head/sys/compat/linux/linux_misc.c == --- head/sys/compat/linux/linux_misc.c Sun Dec 29 15:47:37 2019 (r356176) +++ head/sys/compat/linux/linux_misc.c Sun Dec 29 15:53:55 2019 (r356177) @@ -47,6 +47,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -2288,4 +2289,67 @@ linux_mincore(struct thread *td, struct linux_mincore_ if (args->start & PAGE_MASK) return (EINVAL); return (kern_mincore(td, args->start, args->len, args->vec)); +} + +#defineSYSLOG_TAG "<6>" + +int +linux_syslog(struct thread *td, struct linux_syslog_args *args) +{ + char buf[128], *src, *dst; + u_int seq; + int buflen, error; + + if (args->type != LINUX_SYSLOG_ACTION_READ_ALL) { + linux_msg(td, "syslog unsupported type 0x%x", args->type); + return (EINVAL); + } + + if (args->len < 6) { + td->td_retval[0] = 0; + return (0); + } + + error = priv_check(td, PRIV_MSGBUF); + if (error) + return (error); + + mtx_lock(_lock); + msgbuf_peekbytes(msgbufp, NULL, 0, ); + mtx_unlock(_lock); + + dst = args->buf; + error = copyout(_TAG, dst, sizeof(SYSLOG_TAG)); + /* The -1 is to skip the trailing '\0'. */ + dst += sizeof(SYSLOG_TAG) - 1; + + while (error == 0) { + mtx_lock(_lock); + buflen = msgbuf_peekbytes(msgbufp, buf, sizeof(buf), ); + mtx_unlock(_lock); + + if (buflen == 0) + break; + + for (src = buf; src < buf + buflen && error == 0; src++) { + if (*src == '\0') + continue; + + if (dst >= args->buf + args->len) + goto out; + + error = copyout(src, dst, 1); + dst++; + + if (*src == '\n' && *(src + 1) != '<' && + dst + sizeof(SYSLOG_TAG) < args->buf + args->len) { + error = copyout(_TAG, + dst, sizeof(SYSLOG_TAG)); + dst += sizeof(SYSLOG_TAG) - 1; + } + } + } +out: + td->td_retval[0] = dst - args->buf; + return (error); } Modified: head/sys/compat/linux/linux_misc.h == --- head/sys/compat/linux/linux_misc.h Sun Dec 29 15:47:37 2019 (r356176) +++ head/sys/compat/linux/linux_misc.h Sun Dec 29 15:53:55 2019 (r356177) @@ -149,6 +149,9 @@ extern int stclohz; #defineLINUX_GRND_NONBLOCK 0x0001 #defineLINUX_GRND_RANDOM 0x0002 +/* Linux syslog flags */ +#defineLINUX_SYSLOG_ACTION_READ_ALL3 + #if defined(__amd64__) &&
svn commit: r356176 - in head: lib/geom/nop sys/geom/nop
Author: oshogbo Date: Sun Dec 29 15:47:37 2019 New Revision: 356176 URL: https://svnweb.freebsd.org/changeset/base/356176 Log: gnop: change the "count until fail" option Change the "count_until_fail" option of gnop, now it enables the failing rating instead of setting them to 100%. The original patch introduced the new flag, which sets the fail/rate to 100% after N requests. In some cases, we don't want to have 100% of failure probabilities. We want to start failing at some point. For example, on the early stage, we may like to allow some read/writes requests before having some requests delayed - when we try to mount the partition, or when we are trying to import the pool. Another case may be to check how scrub in ZFS will behave on different stages. This allows us to cover more cases. The previous behavior still may be configured. Reviewed by: kib Differential Revision:https://reviews.freebsd.org/D22632 Modified: head/lib/geom/nop/gnop.8 head/sys/geom/nop/g_nop.c Modified: head/lib/geom/nop/gnop.8 == --- head/lib/geom/nop/gnop.8Sun Dec 29 15:46:59 2019(r356175) +++ head/lib/geom/nop/gnop.8Sun Dec 29 15:47:37 2019(r356176) @@ -123,8 +123,8 @@ See Additional options: .Bl -tag -width "-c count_until_fail" .It Fl c Ar count_until_fail -Specifies the number of I/O requests to allow before setting the read and write -failure probabilities to 100%. +Specifies the number of I/O requests to allow before setting the read, write and +delay failure probabilities. .It Fl d Ar delaymsec Specifies the delay of the requests in milliseconds. Note that requests will be delayed before they are sent to the backing device. Modified: head/sys/geom/nop/g_nop.c == --- head/sys/geom/nop/g_nop.c Sun Dec 29 15:46:59 2019(r356175) +++ head/sys/geom/nop/g_nop.c Sun Dec 29 15:47:37 2019(r356176) @@ -204,31 +204,35 @@ g_nop_start(struct bio *bp) struct bio *cbp; u_int failprob, delayprob, delaytime; - failprob = delayprob = 0; + failprob = delayprob = delaytime = 0; gp = bp->bio_to->geom; sc = gp->softc; G_NOP_LOGREQ(bp, "Request received."); mtx_lock(>sc_lock); - if (sc->sc_count_until_fail != 0 && --sc->sc_count_until_fail == 0) { - sc->sc_rfailprob = 100; - sc->sc_wfailprob = 100; - } switch (bp->bio_cmd) { case BIO_READ: sc->sc_reads++; sc->sc_readbytes += bp->bio_length; - failprob = sc->sc_rfailprob; - delayprob = sc->sc_rdelayprob; - delaytime = sc->sc_delaymsec; + if (sc->sc_count_until_fail != 0) { + sc->sc_count_until_fail -= 1; + } else { + failprob = sc->sc_rfailprob; + delayprob = sc->sc_rdelayprob; + delaytime = sc->sc_delaymsec; + } break; case BIO_WRITE: sc->sc_writes++; sc->sc_wrotebytes += bp->bio_length; - failprob = sc->sc_wfailprob; - delayprob = sc->sc_wdelayprob; - delaytime = sc->sc_delaymsec; + if (sc->sc_count_until_fail != 0) { + sc->sc_count_until_fail -= 1; + } else { + failprob = sc->sc_wfailprob; + delayprob = sc->sc_wdelayprob; + delaytime = sc->sc_delaymsec; + } break; case BIO_DELETE: sc->sc_deletes++; @@ -262,6 +266,7 @@ g_nop_start(struct bio *bp) break; } mtx_unlock(>sc_lock); + if (failprob > 0) { u_int rval; ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356175 - head/sys/compat/linprocfs
Author: trasz Date: Sun Dec 29 15:46:59 2019 New Revision: 356175 URL: https://svnweb.freebsd.org/changeset/base/356175 Log: Make linprocfs(5) provide an empty /proc/modules. This should silence some warnings. MFC after:2 weeks Sponsored by: The FreeBSD Foundation Modified: head/sys/compat/linprocfs/linprocfs.c Modified: head/sys/compat/linprocfs/linprocfs.c == --- head/sys/compat/linprocfs/linprocfs.c Sun Dec 29 15:40:02 2019 (r356174) +++ head/sys/compat/linprocfs/linprocfs.c Sun Dec 29 15:46:59 2019 (r356175) @@ -1504,22 +1504,22 @@ linprocfs_dofilesystems(PFS_FILL_ARGS) return(0); } -#if 0 /* * Filler function for proc/modules */ static int linprocfs_domodules(PFS_FILL_ARGS) { +#if 0 struct linker_file *lf; TAILQ_FOREACH(lf, _files, link) { sbuf_printf(sb, "%-20s%8lu%4d\n", lf->filename, (unsigned long)lf->size, lf->refs); } +#endif return (0); } -#endif /* * Filler function for proc/pid/fd @@ -1713,10 +1713,8 @@ linprocfs_init(PFS_INIT_ARGS) NULL, NULL, NULL, PFS_RD); pfs_create_file(root, "meminfo", _domeminfo, NULL, NULL, NULL, PFS_RD); -#if 0 pfs_create_file(root, "modules", _domodules, NULL, NULL, NULL, PFS_RD); -#endif pfs_create_file(root, "mounts", _domtab, NULL, NULL, NULL, PFS_RD); pfs_create_file(root, "mtab", _domtab, ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356174 - in head: lib/geom/nop sys/geom/nop
Author: oshogbo Date: Sun Dec 29 15:40:02 2019 New Revision: 356174 URL: https://svnweb.freebsd.org/changeset/base/356174 Log: gnop: allow to change the name of created device Thanks to this option we can create more then one gnop provider from single provider. This may be useful for temporary labeling some data on the disk. Reviewed by: markj, allanjude, bcr Differential Revision:https://reviews.freebsd.org/D22304 Modified: head/lib/geom/nop/geom_nop.c head/lib/geom/nop/gnop.8 head/sys/geom/nop/g_nop.c Modified: head/lib/geom/nop/geom_nop.c == --- head/lib/geom/nop/geom_nop.cSun Dec 29 15:39:43 2019 (r356173) +++ head/lib/geom/nop/geom_nop.cSun Dec 29 15:40:02 2019 (r356174) @@ -56,12 +56,13 @@ struct g_command class_commands[] = { { 'w', "wfailprob", "-1", G_TYPE_NUMBER }, { 'x', "wdelayprob", "-1", G_TYPE_NUMBER }, { 'z', "physpath", G_NOP_PHYSPATH_PASSTHROUGH, G_TYPE_STRING }, + { 'Z', "gnopname", G_VAL_OPTIONAL, G_TYPE_STRING }, G_OPT_SENTINEL }, "[-v] [-c count_until_fail] [-d delaymsec] [-e error] [-o offset] " "[-p stripesize] [-P stripeoffset] [-q rdelayprob] [-r rfailprob] " "[-s size] [-S secsize] [-w wfailprob] [-x wdelayprob] " - "[-z physpath] dev ..." + "[-z physpath] [-Z gnopname] dev ..." }, { "configure", G_FLAG_VERBOSE, NULL, { Modified: head/lib/geom/nop/gnop.8 == --- head/lib/geom/nop/gnop.8Sun Dec 29 15:39:43 2019(r356173) +++ head/lib/geom/nop/gnop.8Sun Dec 29 15:40:02 2019(r356174) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd September 13, 2019 +.Dd December 29, 2019 .Dt GNOP 8 .Os .Sh NAME @@ -47,6 +47,7 @@ .Op Fl w Ar wfailprob .Op Fl x Ar wdelayprob .Op Fl z Ar physpath +.Op Fl Z Ar gnopname .Ar dev ... .Nm .Cm configure @@ -153,6 +154,11 @@ Be more verbose. Specifies write delay probability in percent. .It Fl z Ar physpath Physical path of the transparent provider. +.It Fl Z Ar gnopname +The name of the new provider. +The suffix +.Dq .nop +will be appended to the provider name. .El .Sh SYSCTL VARIABLES The following Modified: head/sys/geom/nop/g_nop.c == --- head/sys/geom/nop/g_nop.c Sun Dec 29 15:39:43 2019(r356173) +++ head/sys/geom/nop/g_nop.c Sun Dec 29 15:40:02 2019(r356174) @@ -2,6 +2,7 @@ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2004-2006 Pawel Jakub Dawidek + * Copyright (c) 2019 Mariusz Zaborski * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -30,6 +31,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include #include @@ -81,6 +83,20 @@ struct g_nop_delay { TAILQ_ENTRY(g_nop_delay) dl_next; }; +static bool +g_nop_verify_nprefix(const char *name) +{ + int i; + + for (i = 0; i < strlen(name); i++) { + if (isalpha(name[i]) == 0 && isdigit(name[i]) == 0) { + return (false); + } + } + + return (true); +} + static void g_nop_orphan(struct g_consumer *cp) { @@ -312,17 +328,17 @@ g_nop_access(struct g_provider *pp, int dr, int dw, in static int g_nop_create(struct gctl_req *req, struct g_class *mp, struct g_provider *pp, -int ioerror, u_int count_until_fail, u_int rfailprob, u_int wfailprob, -u_int delaymsec, u_int rdelayprob, u_int wdelayprob, off_t offset, -off_t size, u_int secsize, off_t stripesize, off_t stripeoffset, -const char *physpath) +const char *gnopname, int ioerror, u_int count_until_fail, +u_int rfailprob, u_int wfailprob, u_int delaymsec, u_int rdelayprob, +u_int wdelayprob, off_t offset, off_t size, u_int secsize, off_t stripesize, +off_t stripeoffset, const char *physpath) { struct g_nop_softc *sc; struct g_geom *gp; struct g_provider *newpp; struct g_consumer *cp; char name[64]; - int error; + int error, n; off_t explicitsize; g_topology_assert(); @@ -373,7 +389,22 @@ g_nop_create(struct gctl_req *req, struct g_class *mp, gctl_error(req, "stripeoffset is too big."); return (EINVAL); } - snprintf(name, sizeof(name), "%s%s", pp->name, G_NOP_SUFFIX); + if (gnopname != NULL && !g_nop_verify_nprefix(gnopname)) { + gctl_error(req, "Name %s is invalid.", gnopname); + return (EINVAL); + } + + if (gnopname != NULL) { + n = snprintf(name, sizeof(name), "%s%s", gnopname, + G_NOP_SUFFIX); + } else { +
svn commit: r356173 - head/sys/vm
Author: markj Date: Sun Dec 29 15:39:43 2019 New Revision: 356173 URL: https://svnweb.freebsd.org/changeset/base/356173 Log: Clear queue op flags in vm_page_mvqueue(). This fixes a regression in r356155, introduced at the last minute. In particular, we must clear PGA_REQUEUE_HEAD before inserting into any queue besides PQ_INACTIVE since that operation is implemented only for PQ_INACTIVE. Reported by: pho, Jenkins via lwhsu Modified: head/sys/vm/vm_page.c Modified: head/sys/vm/vm_page.c == --- head/sys/vm/vm_page.c Sun Dec 29 13:54:02 2019(r356172) +++ head/sys/vm/vm_page.c Sun Dec 29 15:39:43 2019(r356173) @@ -4018,6 +4018,7 @@ vm_page_mvqueue(vm_page_t m, const uint8_t nqueue, con if ((old.flags & PGA_DEQUEUE) != 0) break; new = old; + new.flags &= ~PGA_QUEUE_OP_MASK; if (nqueue == PQ_ACTIVE) new.act_count = max(old.act_count, ACT_INIT); if (old.queue == nqueue) { ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356172 - head/sys/compat/linux
Author: trasz Date: Sun Dec 29 13:54:02 2019 New Revision: 356172 URL: https://svnweb.freebsd.org/changeset/base/356172 Log: Make Linux stat(2) et al distinguish between block and character devices. It's required for LTP, among other things. It's not complete, but good enough for now. Reviewed by: kib MFC after:2 weeks Sponsored by: The FreeBSD Foundation Differential Revision:https://reviews.freebsd.org/D22950 Modified: head/sys/compat/linux/linux_stats.c Modified: head/sys/compat/linux/linux_stats.c == --- head/sys/compat/linux/linux_stats.c Sun Dec 29 12:24:41 2019 (r356171) +++ head/sys/compat/linux/linux_stats.c Sun Dec 29 13:54:02 2019 (r356172) @@ -65,6 +65,11 @@ translate_vnhook_major_minor(struct vnode *vp, struct { int major, minor; + if (vn_isdisk(vp, NULL)) { + sb->st_mode &= ~S_IFMT; + sb->st_mode |= S_IFBLK; + } + if (vp->v_type == VCHR && vp->v_rdev != NULL && linux_driver_get_major_minor(devtoname(vp->v_rdev), , ) == 0) { @@ -114,6 +119,10 @@ translate_fd_major_minor(struct thread *td, int fd, st fget(td, fd, _no_rights, ) != 0) return; vp = fp->f_vnode; + if (vp != NULL && vn_isdisk(vp, NULL)) { + buf->st_mode &= ~S_IFMT; + buf->st_mode |= S_IFBLK; + } if (vp != NULL && vp->v_rdev != NULL && linux_driver_get_major_minor(devtoname(vp->v_rdev), , ) == 0) { ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r356171 - head/usr.bin/vmstat
Author: trasz Date: Sun Dec 29 12:24:41 2019 New Revision: 356171 URL: https://svnweb.freebsd.org/changeset/base/356171 Log: Move type casts into a single place. No functional changes. MFC after:2 weeks Modified: head/usr.bin/vmstat/vmstat.c Modified: head/usr.bin/vmstat/vmstat.c == --- head/usr.bin/vmstat/vmstat.cSun Dec 29 12:22:11 2019 (r356170) +++ head/usr.bin/vmstat/vmstat.cSun Dec 29 12:24:41 2019 (r356171) @@ -789,15 +789,14 @@ dovmstat(unsigned int interval, int reps) xo_close_container("processes"); xo_open_container("memory"); #define vmstat_pgtok(a) ((uintmax_t)(a) * (sum.v_page_size >> 10)) -#definerate(x) (((x) * rate_adj + halfuptime) / uptime)/* round */ +#definerate(x) (unsigned long)(((x) * rate_adj + halfuptime) / uptime) if (hflag) { prthuman("available-memory", total.t_avm * (uint64_t)sum.v_page_size, 5, HN_B); prthuman("free-memory", total.t_free * (uint64_t)sum.v_page_size, 5, HN_B); prthuman("total-page-faults", - (unsigned long)rate(sum.v_vm_faults - - osum.v_vm_faults), 5, 0); + rate(sum.v_vm_faults - osum.v_vm_faults), 5, 0); xo_emit(" "); } else { xo_emit(" "); @@ -808,55 +807,48 @@ dovmstat(unsigned int interval, int reps) vmstat_pgtok(total.t_free)); xo_emit(" "); xo_emit("{:total-page-faults/%5lu} ", - (unsigned long)rate(sum.v_vm_faults - - osum.v_vm_faults)); + rate(sum.v_vm_faults - osum.v_vm_faults)); } xo_close_container("memory"); xo_open_container("paging-rates"); xo_emit("{:page-reactivated/%3lu} ", - (unsigned long)rate(sum.v_reactivated - - osum.v_reactivated)); + rate(sum.v_reactivated - osum.v_reactivated)); xo_emit("{:paged-in/%3lu} ", - (unsigned long)rate(sum.v_swapin + sum.v_vnodein - + rate(sum.v_swapin + sum.v_vnodein - (osum.v_swapin + osum.v_vnodein))); xo_emit("{:paged-out/%3lu}", - (unsigned long)rate(sum.v_swapout + sum.v_vnodeout - + rate(sum.v_swapout + sum.v_vnodeout - (osum.v_swapout + osum.v_vnodeout))); if (hflag) { prthuman("freed", - (unsigned long)rate(sum.v_tfree - osum.v_tfree), - 5, 0); + rate(sum.v_tfree - osum.v_tfree), 5, 0); prthuman("scanned", - (unsigned long)rate(sum.v_pdpages - osum.v_pdpages), - 5, 0); + rate(sum.v_pdpages - osum.v_pdpages), 5, 0); xo_emit(" "); } else { xo_emit(" "); xo_emit("{:freed/%5lu} ", - (unsigned long)rate(sum.v_tfree - osum.v_tfree)); + rate(sum.v_tfree - osum.v_tfree)); xo_emit("{:scanned/%4lu} ", - (unsigned long)rate(sum.v_pdpages - osum.v_pdpages)); + rate(sum.v_pdpages - osum.v_pdpages)); } xo_close_container("paging-rates"); devstats(); xo_open_container("fault-rates"); - xo_emit("{:interrupts/%4lu}", - (unsigned long)rate(sum.v_intr - osum.v_intr)); + xo_emit("{:interrupts/%4lu}", rate(sum.v_intr - osum.v_intr)); if (hflag) { prthuman("system-calls", - (unsigned long)rate(sum.v_syscall - osum.v_syscall), - 5, 0); + rate(sum.v_syscall - osum.v_syscall), 5, 0); prthuman("context-switches", - (unsigned long)rate(sum.v_swtch - osum.v_swtch), - 5, 0); + rate(sum.v_swtch - osum.v_swtch), 5, 0); } else { xo_emit(" "); xo_emit("{:system-calls/%5lu} " "{:context-switches/%5lu}", - (unsigned long)rate(sum.v_syscall - osum.v_syscall), - (unsigned long)rate(sum.v_swtch - osum.v_swtch)); + rate(sum.v_syscall -
svn commit: r356170 - head/sys/compat/linux
Author: trasz Date: Sun Dec 29 12:22:11 2019 New Revision: 356170 URL: https://svnweb.freebsd.org/changeset/base/356170 Log: Implement Linux BLKGETSIZE64 ioctl. MFC after:2 weeks Sponsored by: The FreeBSD Foundation Modified: head/sys/compat/linux/linux_ioctl.c head/sys/compat/linux/linux_ioctl.h Modified: head/sys/compat/linux/linux_ioctl.c == --- head/sys/compat/linux/linux_ioctl.c Sun Dec 29 06:59:09 2019 (r356169) +++ head/sys/compat/linux/linux_ioctl.c Sun Dec 29 12:22:11 2019 (r356170) @@ -278,6 +278,7 @@ linux_ioctl_disk(struct thread *td, struct linux_ioctl struct file *fp; int error; u_int sectorsize; + uint64_t blksize64; off_t mediasize; error = fget(td, args->fd, _ioctl_rights, ); @@ -300,6 +301,15 @@ linux_ioctl_disk(struct thread *td, struct linux_ioctl return (copyout(, (void *)args->arg, sizeof(sectorsize))); break; + case LINUX_BLKGETSIZE64: + error = fo_ioctl(fp, DIOCGMEDIASIZE, + (caddr_t), td->td_ucred, td); + fdrop(fp, td); + if (error) + return (error); + blksize64 = mediasize;; + return (copyout(, (void *)args->arg, + sizeof(blksize64))); case LINUX_BLKSSZGET: error = fo_ioctl(fp, DIOCGSECTORSIZE, (caddr_t), td->td_ucred, td); Modified: head/sys/compat/linux/linux_ioctl.h == --- head/sys/compat/linux/linux_ioctl.h Sun Dec 29 06:59:09 2019 (r356169) +++ head/sys/compat/linux/linux_ioctl.h Sun Dec 29 12:22:11 2019 (r356170) @@ -57,9 +57,10 @@ #defineLINUX_BLKSECTSET0x1266 #defineLINUX_BLKSECTGET0x1267 #defineLINUX_BLKSSZGET 0x1268 +#defineLINUX_BLKGETSIZE64 0x1272 #define LINUX_IOCTL_DISK_MINLINUX_BLKROSET -#define LINUX_IOCTL_DISK_MAXLINUX_BLKSSZGET +#define LINUX_IOCTL_DISK_MAXLINUX_BLKGETSIZE64 /* * hdio ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"