Re: [zfs-discuss] How to delete hundreds of emtpy snapshots
I got overzealous with snapshot creation. Every 5 mins is a bad idea. Way too many. What's the easiest way to delete the empty ones? zfs list takes FOREVER You might enjoy reading: ZFS snapshot massacre http://blogs.sun.com/chrisg/entry/zfs_snapshot_massacre. (Yes, the . is part of the URL (NMF) - so add it or you'll 404). Rob This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to delete hundreds of emtpy snapshots
Also http://blogs.sun.com/chrisg/entry/a_faster_zfs_snapshot_massacre which I run every night. Lots of snapshots are not a bad thing it is keeping them for a long time that takes space. I'm still snapping every 10 minutes and it is great. The thing I discovered was that I really wanted to be able to find distinct verstons fo a file so that I could see which one was the version I wanted to get back. To that end I wrote http://blogs.sun.com/chrisg/entry/zfs_versions_of_a_file and filed this RFE to help with this: http://bugs.opensolaris.org/view_bug.do?bug_id=6719101 This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to delete hundreds of emtpy snapshots
I don't know if there's any easy way to delete them, but I can offer a suggestion for the future. Tim Foster has a ZFS automatic snapshot script that would be worth taking a look at: http://blogs.sun.com/timf/entry/zfs_automatic_snapshots_0_10 That script lets you fairly easily set up snapshots so you have say: 12x 5 minute snapshots 24x hourly snapshots 7x daily snapshots 5x weekly snapshots 12x monthly snapshots It's a grand total of 60 snapshots, you still get backups every 5 minutes for anything you've done recently, but you can go back a whole year if you really need to find something. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to delete hundreds of emtpy snapshots
zfs list is mighty slow on systems with a large number of objects, but there is no foreseeable plan that I'm aware of to solve that problem. Never the less, you need to do a zfs list, therefore, do it once and work from that. zfs list /tmp/zfs.out for i in `grep mydataset@ /tmp/zfs.out`; do zfs destroy $i; done As for 5 minute snapshots this is NOT a bad idea. It is, however, complex to manage. Thus, you need to employ tactics to make it more digestible. You need to ask yourself first why you want 5 min snaps. Is it replication? If so, create it, replicate it, destroy all but the last snapshot or even rotate them. Or, is it fallback in case you make a mistake? Then just keep around the last 6 snapshots or so. zfs rename zfs destroy are your friends use them wisely. :) If you want to discuss exactly what your trying to facilitate I'm sure we can come up with some more concrete ideas to help you. benr. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to delete hundreds of emtpy snapshots
Sylvain Dusart schrieb:
Hi Joe,
I use this script to delete my empty snapshots :
#!/bin/bash
NOW=$(date +%Y%m%d-%H%M%S)
POOL=tank
zfs snapshot -r [EMAIL PROTECTED]
FS_WITH_SNAPSHOTS=$(zfs list -t snapshot | grep '@' | cut -d '@' -f 1 | uniq)
for fs in $FS_WITH_SNAPSHOTS ; do
EMPTY_SNAPSHOTS=$(zfs list -t snapshot | grep -v @$NOW |
grep ${fs}@ | awk ' $2 == 0 { print $1 }' )
for snapshot in $EMPTY_SNAPSHOTS ; do
echo Destroying empty snapshot $snapshot
zfs destroy $snapshot
done
done
Hope that helps,
This script is broken - because the zfs list output is broken. You
might end up deleting snapshots with data in it.
Data which is in more than one snapshot will not get counted in the zfs
list output at all.
Let's try it with an example:
# zfs create export/test
# mkfile 10m /export/test/bla
# zfs snapshot export/[EMAIL PROTECTED]
# zfs snapshot export/[EMAIL PROTECTED]
# rm /export/test/bla
Now both snapshots should contain (the same) 10MB of valuable data (the
file /export/test/bla just removed). But:
# zfs list -r export/test
NAME USED AVAIL REFER MOUNTPOINT
export/test 24.5K 171G 24.5K /export/test
export/[EMAIL PROTECTED] 0 - 24.5K -
export/[EMAIL PROTECTED] 0 - 24.5K -
... indicates two empty snapshots. But after removing one of the
snapshots ...
# zfs destroy export/[EMAIL PROTECTED]
... the data magically shows up on the second snapshot (in my case not
10MB, because the compression setting was inherited from the parent zfs)
# zfs list -r export/test
NAME USED AVAIL REFER MOUNTPOINT
export/test47K 171G 24.5K /export/test
export/[EMAIL PROTECTED] 22.5K - 24.5K -
Your script above would have deleted both snapshots. One possible
solution would be to re-evaluate the zfs list output after each
snapshot deleted. This way the last one of data-identical snapshots
would be preserved. Something like:
while :; do
EMPTYSNAPSHOT=$(zfs list -Ht snapshot -o name,used | \
awk '$2 == 0 { print $1; exit }')
[ -n $EMPTYSNAPSHOT ] || break
zfs destroy $EMPTYSNAPSHOT
done
Daniel
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to delete hundreds of emtpy snapshots
On Thu, 17 Jul 2008, Ben Rockwood wrote: zfs list is mighty slow on systems with a large number of objects, but there is no foreseeable plan that I'm aware of to solve that problem. Never the less, you need to do a zfs list, therefore, do it once and work from that. If the snapshots were done from a script then their names are easily predictable and similar logic can be used to re-create the existing names. This avoids the need to do a 'zfs list'. Bob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to delete hundreds of emtpy snapshots
In one of my prior experiments, I included the names of the snapshots I created in a plain text file. I used this file, and not the zfs list output to determine which snapshots I was going to remove when it came time. I don't even remember *why* I did that in the first place, but it certainly made things easier when it came time to clean up a whole bunch of stuff... (And was not impacted by zfs list being non-snappy...) The snapshot naming scheme meant that it was dead easy to work out which to remove / keep... Right now, I don't have a system (that box was killed in a dreadful xen experiment :) so I'll be watching this thread with renewed interest to see who else is doing what... Nathan. Bob Friesenhahn wrote: On Thu, 17 Jul 2008, Ben Rockwood wrote: zfs list is mighty slow on systems with a large number of objects, but there is no foreseeable plan that I'm aware of to solve that problem. Never the less, you need to do a zfs list, therefore, do it once and work from that. If the snapshots were done from a script then their names are easily predictable and similar logic can be used to re-create the existing names. This avoids the need to do a 'zfs list'. Bob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] How to delete hundreds of emtpy snapshots
I got overzealous with snapshot creation. Every 5 mins is a bad idea. Way too many. What's the easiest way to delete the empty ones? zfs list takes FOREVER ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to delete hundreds of emtpy snapshots [SEC=PERSONAL]
Hi there, Yes 5 minutes is too often. I would, if it is easy and you have enough room, and don't want any of the snapshots, create another temp zfs file system and copy your data there and destroy the current zfs file system. Then recreate it and copy the data back. I don't know if there is a better way ... If there is I would be interested to know incase I ever actually need to fix something like this :) Ta, --- Cooper Ry Lees UNIX Evangelist Information Management Services (IMS) Australian Nuclear Science and Technology Organisation (ANSTO) [p] +61 2 9717 3853 [e] [EMAIL PROTECTED] On 17/07/2008, at 3:16 PM, Joe S wrote: I got overzealous with snapshot creation. Every 5 mins is a bad idea. Way too many. What's the easiest way to delete the empty ones? zfs list takes FOREVER ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
