When running this commit on OsX, which is a little more strict with mutex 
release; we get this situation:

```
    frame #4:  kernel.development`panic(mutex_enter on destroyed mutex) at 
debug.c:605 [opt]
    frame #5:  spl`spl_mutex_enter(mp=0xffffff886f557b00, file="aggsum.c", 
line=190) at spl-mutex.c:294 [opt]
    frame #6:  zfs`aggsum_add(as=0xffffff7f8e832370, delta=-360) at 
aggsum.c:190 [opt]
    frame #7:  zfs`arc_space_return(space=360, type=ARC_SPACE_HDRS) at arc.c:0 
[opt]
    frame #8:  spl`kmem_magazine_destroy(cp=0xffffff886f546c70, 
mp=<unavailable>, nrounds=<unavailable>) at spl-kmem.c:1655 [opt]
    frame #9:  spl`kmem_cache_magazine_purge(cp=0xffffff886f546c70) at 
spl-kmem.c:2870 [opt]
    frame #10:  spl`kmem_cache_destroy(cp=0xffffff886f546c70) at 
spl-kmem.c:3831 [opt]
    frame #11:  zfs`arc_fini [inlined] buf_fini at arc.c:1301 [opt]
  * frame #12:  zfs`arc_fini at arc.c:7822 [opt]
```

Which at a glance I _think_ is due to that the release order is:
```
frame #12: 0xffffff7f8e5ea342 zfs`arc_fini at arc.c:7822 [opt]
   7819         cv_destroy(&arc_reclaim_waiters_cv);
   7820
   7821         arc_state_fini();     # aggsums released in here
-> 7822         buf_fini();
```

but `buf_fini()` can trigger release of magazines, which has callbacks to 
`arc_space_return()` which attempts to use the aggsums released in prior call 
to `arc_stat_fini()`.

Moving the aggsums release to
```
        arc_state_fini();
        buf_fini();

        aggsum_fini(&arc_meta_used);
        aggsum_fini(&arc_size);
        aggsum_fini(&astat_data_size);
        aggsum_fini(&astat_metadata_size);
        aggsum_fini(&astat_hdr_size);
        aggsum_fini(&astat_other_size);
        aggsum_fini(&astat_l2_hdr_size);
```

lets me unload that one time I tried it at least :)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/679#issuecomment-414934847
------------------------------------------
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Tc438439e218e9400-Mf819d7e4020516894b3b00fe
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

Reply via email to