On Thu, Aug 30, 2018 at 09:55:27AM +0300, Paul wrote:
> 30 August 2018, 00:22:14, by "Mark Johnston" <ma...@freebsd.org>:
>
> > On Wed, Aug 29, 2018 at 12:42:33PM +0300, Paul wrote:
> > > Hello team,
> > >
> > >
> > > It seems like a commit on Mar 23 introduced a bug: if during execution of
> > > arc_adjust()
> > > target is reached after MRU is evicted current code continues evicting
> > > MFU. Before said
> > > commit, on the step prior to MFU eviction, target value was recalculated
> > > as:
> > >
> > > target = arc_size - arc_c;
> > >
> > > arc_size here is a global variable that was being updated accordingly,
> > > during MRU eviction,
> > > hence this expression, resulted in zero or negative target if MRU
> > > eviction was enough
> > > to reach the original goal.
> > >
> > > Modern version uses cached value of arc_size, called asize:
> > >
> > > target = asize - arc_c;
> > >
> > > Because asize stays constant during execution of whole body of
> > > arc_adjust() it means that
> > > above expression will always be evaluated to value > 0, causing MFU to be
> > > evicted every
> > > time, even if MRU eviction has reached the goal already. Because of the
> > > difference in
> > > nature of MFU and MRU, globally it leads to eventual reduction of amount
> > > of MFU in ARC
> > > to dramatic numbers.
> >
> > Hi Paul,
> >
> > Your analysis does seem right to me. I cc'ed the openzfs mailing list
> > so that an actual ZFS expert can chime in; it looks like this behaviour
> > is consistent between FreeBSD, illumos and ZoL.
> >
> > Have you already tried the obvious "fix" of subtracting total_evicted
> > from the MFU target?
>
> We are going to apply the asize patch (plus the ameta, as suggested by
> Richard) and reboot
> one of our production servers this night or the following.
Just to be explicit, are you testing something equivalent to the patch
at the end of this email?
> Then we have to wait a few days and observer the ARC behaviour.
Thanks! Please let us know how it goes: we're preparing to release
FreeBSD 12.0 shortly and I'd like to get this fixed in head/ as soon as
possible.
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
index 1387925c4607..882c04dba50a 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
@@ -4446,6 +4446,12 @@ arc_adjust(void)
arc_adjust_impl(arc_mru, 0, target, ARC_BUFC_METADATA);
}
+ /*
+ * Re-sum ARC stats after the first round of evictions.
+ */
+ asize = aggsum_value(&arc_size);
+ ameta = aggsum_value(&arc_meta_used);
+
/*
* Adjust MFU size
*
------------------------------------------
openzfs: openzfs-developer
Permalink:
https://openzfs.topicbox.com/groups/developer/T10a105c53bcce15c-M1c45cd09114d2ce2e8c9dd26
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription
