Author: markj Date: Mon Oct 19 16:55:03 2020 New Revision: 366839 URL: https://svnweb.freebsd.org/changeset/base/366839
Log: uma: Avoid depleting keg reserves when filling a bucket zone_import() fetches a free or partially free slab from the keg and then uses its items to populate an array, typically filling a bucket. If a single allocation causes the keg to drop below its minimum reserve, the inner loop ends. However, if the bucket is still not full and M_USE_RESERVE is specified, the outer loop will continue to fetch items from the keg. If M_USE_RESERVE is specified and the number of free items is below the reserved limit, we should return only a single item. Otherwise, if the bucket size is larger than the reserve, all of the reserved items may end up in a single per-CPU bucket, invisible to other CPUs. Reviewed by: rlibby MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D26771 Modified: head/sys/vm/uma_core.c Modified: head/sys/vm/uma_core.c ============================================================================== --- head/sys/vm/uma_core.c Mon Oct 19 16:54:06 2020 (r366838) +++ head/sys/vm/uma_core.c Mon Oct 19 16:55:03 2020 (r366839) @@ -3734,10 +3734,17 @@ zone_import(void *arg, void **bucket, int max, int dom stripe = howmany(max, vm_ndomains); #endif dom = &keg->uk_domain[slab->us_domain]; - while (slab->us_freecount && i < max) { + do { bucket[i++] = slab_alloc_item(keg, slab); - if (dom->ud_free_items <= keg->uk_reserve) - break; + if (dom->ud_free_items <= keg->uk_reserve) { + /* + * Avoid depleting the reserve after a + * successful item allocation, even if + * M_USE_RESERVE is specified. + */ + KEG_UNLOCK(keg, slab->us_domain); + goto out; + } #ifdef NUMA /* * If the zone is striped we pick a new slab for every @@ -3751,13 +3758,14 @@ zone_import(void *arg, void **bucket, int max, int dom vm_ndomains > 1 && --stripe == 0) break; #endif - } + } while (slab->us_freecount != 0 && i < max); KEG_UNLOCK(keg, slab->us_domain); + /* Don't block if we allocated any successfully. */ flags &= ~M_WAITOK; flags |= M_NOWAIT; } - +out: return i; } _______________________________________________ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
