Re: zpool frag
Allan Jude wrote this message on Sun, Sep 21, 2014 at 11:06 -0400: > On 2014-09-21 04:57, Beeblebrox wrote: > > FRAG means fragmentation, right? Zpool fragmentation? That's news to me. If > > this is real how do I fix it? > > > > NAME SIZE ALLOC FREE FRAG EXPANDSZCAP DEDUP HEALTH ALTROOT > > pool1 75.5G 53.7G 21.8G60% -71% 1.00x ONLINE - > > pool2 48.8G 26.2G 22.6G68% -53% 1.00x ONLINE - > > pool3 204G 177G 27.0G53% -86% 1.11x ONLINE - > > > > Regards. > > > > > > > > - > > FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS > > -- > > View this message in context: > > http://freebsd.1045724.n5.nabble.com/zpool-frag-tp5950788.html > > Sent from the freebsd-current mailing list archive at Nabble.com. > > ___ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > > > It is not something you 'fix', it is just a metric to help you > understand the performance of your pool. The higher the fragmentation, > the longer it might take to allocate new space, and obviously you will > have more random seek time while reading from the pool. > > As Steven mentions, there is no defragmentation tool for ZFS. You can > zfs send/recv or backup/restore the pool if you have a strong enough > reason to want to get the fragmentation number down. > > It is a fairly natural side effect of a copy-on-write file system. > > Note: the % is not the % fragmented, IIRC, it is the percentage of the > free blocks that are less that a specific size. I forget what that size is. Can we get this documented in the zpool man page? I assume that the FRAG is the same as: fragmentation The amount of fragmentation in the pool. and that description is woefully lacking.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: zpool frag
On 2014-09-21 13:39, Peter Wemm wrote: > On Sunday, September 21, 2014 06:12:09 PM Steven Hartland wrote: >> - Original Message - >> >>> From: "Peter Wemm" >>> >>> On Sunday, September 21, 2014 11:06:10 AM Allan Jude wrote: On 2014-09-21 04:57, Beeblebrox wrote: > FRAG means fragmentation, right? Zpool fragmentation? That's news to > me. > If > this is real how do I fix it? > > NAME SIZE ALLOC FREE FRAG EXPANDSZCAP DEDUP HEALTH > ALTROOT pool1 75.5G 53.7G 21.8G60% -71% 1.00x > ONLINE - pool2 48.8G 26.2G 22.6G68% -53% > 1.00x > ONLINE - pool3 204G 177G 27.0G53% -86% > 1.11x > ONLINE - It is not something you 'fix', it is just a metric to help you understand the performance of your pool. The higher the fragmentation, the longer it might take to allocate new space, and obviously you will have more random seek time while reading from the pool. As Steven mentions, there is no defragmentation tool for ZFS. You can zfs send/recv or backup/restore the pool if you have a strong enough reason to want to get the fragmentation number down. It is a fairly natural side effect of a copy-on-write file system. Note: the % is not the % fragmented, IIRC, it is the percentage of the free blocks that are less that a specific size. I forget what that size is. >>> >>> I fear that the information presented in its current form is going to >>> generate lots of fear and confusion. >>> >>> The other thing to consider is that this gets much, much worse as the pool >>> fills up. Even UFS has issues with fragmentation when it fills, but ZFS >>> is far more sensative to it. In the freebsd.org cluster we have a health >>> check alert at 80% full, but even that's probably on the high side. >> >> This "should" be less of an issue if you have the spacemap_histogram feature >> enabled on the pool, which IIRC if your seeing FRAG details should be the >> case. > > Hopefully so. The catch though is when its been run without it until > recently > it can be a bit of a surprise. > > From an email exchange with George Wilson (developer of the spacemap_histogram feature): "You can use it on existing pools but the histogram only gets created when you condense a space_map (a process by which ZFS tries to make the space_map ondisk smaller). This means that when you look at an existing pool it may have some space_maps with histograms and ones without." This likely means that on a pool that has (recently??) been upgrade, the FRAG metric may fluctuate wildly, as more and more space_maps get a histogram. The histogram feature doesn't really solve the problem of fragmentation, it just makes the pool perform better when it is fragmented, as the pool can more reliably find the space_map containing the largest area of contiguous free space in order to maintain performance. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: zpool frag
On Sunday, September 21, 2014 06:12:09 PM Steven Hartland wrote: > - Original Message - > > > From: "Peter Wemm" > > > > On Sunday, September 21, 2014 11:06:10 AM Allan Jude wrote: > > > On 2014-09-21 04:57, Beeblebrox wrote: > > > > FRAG means fragmentation, right? Zpool fragmentation? That's news to > > > > me. > > > > If > > > > this is real how do I fix it? > > > > > > > > NAME SIZE ALLOC FREE FRAG EXPANDSZCAP DEDUP HEALTH > > > > ALTROOT pool1 75.5G 53.7G 21.8G60% -71% 1.00x > > > > ONLINE - pool2 48.8G 26.2G 22.6G68% -53% > > > > 1.00x > > > > ONLINE - pool3 204G 177G 27.0G53% -86% > > > > 1.11x > > > > ONLINE - > > > > > > It is not something you 'fix', it is just a metric to help you > > > understand the performance of your pool. The higher the fragmentation, > > > the longer it might take to allocate new space, and obviously you will > > > have more random seek time while reading from the pool. > > > > > > As Steven mentions, there is no defragmentation tool for ZFS. You can > > > zfs send/recv or backup/restore the pool if you have a strong enough > > > reason to want to get the fragmentation number down. > > > > > > It is a fairly natural side effect of a copy-on-write file system. > > > > > > Note: the % is not the % fragmented, IIRC, it is the percentage of the > > > free blocks that are less that a specific size. I forget what that size > > > is. > > > > I fear that the information presented in its current form is going to > > generate lots of fear and confusion. > > > > The other thing to consider is that this gets much, much worse as the pool > > fills up. Even UFS has issues with fragmentation when it fills, but ZFS > > is far more sensative to it. In the freebsd.org cluster we have a health > > check alert at 80% full, but even that's probably on the high side. > > This "should" be less of an issue if you have the spacemap_histogram feature > enabled on the pool, which IIRC if your seeing FRAG details should be the > case. Hopefully so. The catch though is when its been run without it until recently it can be a bit of a surprise. -- Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246 signature.asc Description: This is a digitally signed message part.
Re: zpool frag
- Original Message - From: "Peter Wemm" On Sunday, September 21, 2014 11:06:10 AM Allan Jude wrote: > On 2014-09-21 04:57, Beeblebrox wrote: > > FRAG means fragmentation, right? Zpool fragmentation? That's news to me. > > If > > this is real how do I fix it? > > > > NAME SIZE ALLOC FREE FRAG EXPANDSZCAP DEDUP HEALTH > > ALTROOT pool1 75.5G 53.7G 21.8G60% -71% 1.00x > > ONLINE - pool2 48.8G 26.2G 22.6G68% -53% 1.00x > > ONLINE - pool3 204G 177G 27.0G53% -86% 1.11x > > ONLINE - > It is not something you 'fix', it is just a metric to help you > understand the performance of your pool. The higher the fragmentation, > the longer it might take to allocate new space, and obviously you will > have more random seek time while reading from the pool. > > As Steven mentions, there is no defragmentation tool for ZFS. You can > zfs send/recv or backup/restore the pool if you have a strong enough > reason to want to get the fragmentation number down. > > It is a fairly natural side effect of a copy-on-write file system. > > Note: the % is not the % fragmented, IIRC, it is the percentage of the > free blocks that are less that a specific size. I forget what that size is. I fear that the information presented in its current form is going to generate lots of fear and confusion. The other thing to consider is that this gets much, much worse as the pool fills up. Even UFS has issues with fragmentation when it fills, but ZFS is far more sensative to it. In the freebsd.org cluster we have a health check alert at 80% full, but even that's probably on the high side. This "should" be less of an issue if you have the spacemap_histogram feature enabled on the pool, which IIRC if your seeing FRAG details should be the case. Regards Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: zpool frag
On Sunday, September 21, 2014 11:06:10 AM Allan Jude wrote: > On 2014-09-21 04:57, Beeblebrox wrote: > > FRAG means fragmentation, right? Zpool fragmentation? That's news to me. > > If > > this is real how do I fix it? > > > > NAME SIZE ALLOC FREE FRAG EXPANDSZCAP DEDUP HEALTH > > ALTROOT pool1 75.5G 53.7G 21.8G60% -71% 1.00x > > ONLINE - pool2 48.8G 26.2G 22.6G68% -53% 1.00x > > ONLINE - pool3 204G 177G 27.0G53% -86% 1.11x > > ONLINE - > > > > Regards. > > > > > > > > - > > FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS > > -- > > View this message in context: > > http://freebsd.1045724.n5.nabble.com/zpool-frag-tp5950788.html Sent from > > the freebsd-current mailing list archive at Nabble.com. > > ___ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > It is not something you 'fix', it is just a metric to help you > understand the performance of your pool. The higher the fragmentation, > the longer it might take to allocate new space, and obviously you will > have more random seek time while reading from the pool. > > As Steven mentions, there is no defragmentation tool for ZFS. You can > zfs send/recv or backup/restore the pool if you have a strong enough > reason to want to get the fragmentation number down. > > It is a fairly natural side effect of a copy-on-write file system. > > Note: the % is not the % fragmented, IIRC, it is the percentage of the > free blocks that are less that a specific size. I forget what that size is. I fear that the information presented in its current form is going to generate lots of fear and confusion. The other thing to consider is that this gets much, much worse as the pool fills up. Even UFS has issues with fragmentation when it fills, but ZFS is far more sensative to it. In the freebsd.org cluster we have a health check alert at 80% full, but even that's probably on the high side. -- Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246 signature.asc Description: This is a digitally signed message part.
Re: zpool frag
On 2014-09-21 04:57, Beeblebrox wrote: > FRAG means fragmentation, right? Zpool fragmentation? That's news to me. If > this is real how do I fix it? > > NAME SIZE ALLOC FREE FRAG EXPANDSZCAP DEDUP HEALTH ALTROOT > pool1 75.5G 53.7G 21.8G60% -71% 1.00x ONLINE - > pool2 48.8G 26.2G 22.6G68% -53% 1.00x ONLINE - > pool3 204G 177G 27.0G53% -86% 1.11x ONLINE - > > Regards. > > > > - > FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS > -- > View this message in context: > http://freebsd.1045724.n5.nabble.com/zpool-frag-tp5950788.html > Sent from the freebsd-current mailing list archive at Nabble.com. > ___ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > It is not something you 'fix', it is just a metric to help you understand the performance of your pool. The higher the fragmentation, the longer it might take to allocate new space, and obviously you will have more random seek time while reading from the pool. As Steven mentions, there is no defragmentation tool for ZFS. You can zfs send/recv or backup/restore the pool if you have a strong enough reason to want to get the fragmentation number down. It is a fairly natural side effect of a copy-on-write file system. Note: the % is not the % fragmented, IIRC, it is the percentage of the free blocks that are less that a specific size. I forget what that size is. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: zpool frag
Backup the pool and restore it is the only way I'm aware of. - Original Message - From: "Beeblebrox" To: Sent: Sunday, September 21, 2014 9:57 AM Subject: zpool frag FRAG means fragmentation, right? Zpool fragmentation? That's news to me. If this is real how do I fix it? NAME SIZE ALLOC FREE FRAG EXPANDSZCAP DEDUP HEALTH ALTROOT pool1 75.5G 53.7G 21.8G60% -71% 1.00x ONLINE - pool2 48.8G 26.2G 22.6G68% -53% 1.00x ONLINE - pool3 204G 177G 27.0G53% -86% 1.11x ONLINE - Regards. - FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/zpool-frag-tp5950788.html Sent from the freebsd-current mailing list archive at Nabble.com. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
