Re: arc_reclaim_thread running hot
On 9/13/18 8:18 AM, Eric van Gyzen wrote:
This morning, I found the arc_reclaim_thread running hot on my laptop
running 12.0-ALPHA5 r338572.
vfs.zfs.arc_max="4294967296" <-- 4 GiB
last pid: 13288; load averages: 1.32, 1.26, 1.16
Mem: 456M Active, 3837M Inact, 743M Laundry, 2563M Wired, 167M Free
ARC: 1131M Total, 304M MFU, 145M MRU, 1344K Anon, 9116K Header, 671M Other
89M Compressed, 361M Uncompressed, 4.03:1 Ratio
22 root -8 - 0 256K CPU2 2 309:20 99.75%
zfskern{arc_reclaim_thread}
zfs_arc_meta_strategy is still the default of 1.
I sampled the thread's stacks with
for N in `jot 1000`; do procstat -kk 100101; done | grep 100101
and put the results here:
https://people.freebsd.org/~vangyzen/arc_reclaim_thread_stacks.txt
I'm happy to help debug this. Just let me know what you need.
The thread eventually returned to normal, and I've rebooted the system
since then, so I no longer have any state. The thread started running
hot at 03:00, so maybe the daily periodic run triggered it. I have to
work now, but I'll try running the periodic stuff manually when I have time.
Eric
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: arc_reclaim_thread running hot
On Thu, 13 Sep 2018 at 15:19, Eric van Gyzen wrote:
>
> This morning, I found the arc_reclaim_thread running hot on my laptop
> running 12.0-ALPHA5 r338572.
>
> vfs.zfs.arc_max="4294967296" <-- 4 GiB
>
> last pid: 13288; load averages: 1.32, 1.26, 1.16
>
> Mem: 456M Active, 3837M Inact, 743M Laundry, 2563M Wired, 167M Free
> ARC: 1131M Total, 304M MFU, 145M MRU, 1344K Anon, 9116K Header, 671M Other
> 89M Compressed, 361M Uncompressed, 4.03:1 Ratio
>
> 22 root -8- 0 256K CPU2 2 309:20 99.75%
> zfskern{arc_reclaim_thread}
>
> zfs_arc_meta_strategy is still the default of 1.
>
> I sampled the thread's stacks with
>
> for N in `jot 1000`; do procstat -kk 100101; done | grep 100101
>
> and put the results here:
>
> https://people.freebsd.org/~vangyzen/arc_reclaim_thread_stacks.txt
>
> I'm happy to help debug this. Just let me know what you need.
>
> Eric
Just to summarize, these are the top 15 active
(guessing that the leftmost function is "current"):
22 arc_space_return
22 uma_dbg_free
26 uma_dbg_alloc
30 mi_switch
31 uma_zalloc_arg
33 multilist_sublist_lock
34 __mtx_unlock_flags
39
46 _sx_xlock
48 hdr_full_cons
50 uma_zfree_arg
58 aggsum_add
91 arc_adjust
156 _sx_xunlock
185 arc_evict_state
And the top 15 in total
(that is, no matter where in the call stack):
44 multilist_sublist_lock
44 uma_dbg_free
46 _sx_xlock
47 uma_dbg_alloc
61 arc_space_return
127 aggsum_add
139 hdr_full_cons
156 _sx_xunlock
211 uma_zfree_arg
219 uma_zalloc_arg
833 arc_evict_state
926 arc_adjust
961 arc_reclaim_thread
961 fork_exit
961 fork_trampoline
(I have nothing sensible to add about the actual problem.)
--
Daniel Nebdal
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
arc_reclaim_thread running hot
This morning, I found the arc_reclaim_thread running hot on my laptop
running 12.0-ALPHA5 r338572.
vfs.zfs.arc_max="4294967296" <-- 4 GiB
last pid: 13288; load averages: 1.32, 1.26, 1.16
Mem: 456M Active, 3837M Inact, 743M Laundry, 2563M Wired, 167M Free
ARC: 1131M Total, 304M MFU, 145M MRU, 1344K Anon, 9116K Header, 671M Other
89M Compressed, 361M Uncompressed, 4.03:1 Ratio
22 root -8- 0 256K CPU2 2 309:20 99.75%
zfskern{arc_reclaim_thread}
zfs_arc_meta_strategy is still the default of 1.
I sampled the thread's stacks with
for N in `jot 1000`; do procstat -kk 100101; done | grep 100101
and put the results here:
https://people.freebsd.org/~vangyzen/arc_reclaim_thread_stacks.txt
I'm happy to help debug this. Just let me know what you need.
Eric
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
