Re: ZFS Snapshots Not able to be accessed under .zfs/snapshot/name
On 08/16/2013 8:49 am, dweimer wrote: On 08/15/2013 10:00 am, dweimer wrote: On 08/14/2013 9:43 pm, Shane Ambler wrote: On 14/08/2013 22:57, dweimer wrote: I have a few systems running on ZFS with a backup script that creates snapshots, then backs up the .zfs/snapshot/name directory to make sure open files are not missed. This has been working great but all of the sudden one of my systems has stopped working. It takes the snapshots fine, zfs list -t spnapshot shows the snapshots, but if you do an ls command, on the .zfs/snapshot/ directory it returns not a directory. part of the zfs list output: NAMEUSED AVAIL REFER MOUNTPOINT zroot 4.48G 29.7G31K none zroot/ROOT 2.92G 29.7G31K none zroot/ROOT/91p5-20130812 2.92G 29.7G 2.92G legacy zroot/home 144K 29.7G 122K /home part of the zfs list -t snapshot output: NAMEUSED AVAIL REFER MOUNTPOINT zroot/ROOT/91p5-20130812@91p5-20130812--bsnap 340K - 2.92G - zroot/home@home--bsnap 22K - 122K - ls /.zfs/snapshot/91p5-20130812--bsnap/ Does work at the right now, since the last reboot, but wasn't always working, this is my boot environment. if I do ls /home/.zfs/snapshot/, result is: ls: /home/.zfs/snapshot/: Not a directory if I do ls /home/.zfs, result is: ls: snapshot: Bad file descriptor shares I have tried zpool scrub zroot, no errors were found, if I reboot the system I can get one good backup, then I start having problems. Anyone else ever ran into this, any suggestions as to a fix? System is running FreeBSD 9.1-RELEASE-p5 #1 r253764: Mon Jul 29 15:07:35 CDT 2013, zpool is running version 28, zfs is running version 5 I can say I've had this problem. Not certain what fixed it. I do remember I decided to stop snapshoting if I couldn't access them and deleted existing snapshots. I later restarted the machine before I went back for another look and they were working. So my guess is a restart without existing snapshots may be the key. Now if only we could find out what started the issue so we can stop it happening again. I had actually rebooted it last night, prior to seeing this message, I do know it didn't have any snapshots this time. As I am booting from ZFS using boot environments I may have had an older boot environment still on the system the last time it was rebooted. Backups ran great last night after the reboot, and I was able to kick off my pre-backup job and access all the snapshots today. Hopefully it doesn't come back, but if it does I will see if I can find anything else wrong. FYI, It didn't shutdown cleanly, so if this helps anyone find the issue, this is from my system logs: Aug 14 22:08:04 cblproxy1 kernel: Aug 14 22:08:04 cblproxy1 kernel: Fatal trap 12: page fault while in kernel mode Aug 14 22:08:04 cblproxy1 kernel: cpuid = 0; apic id = 00 Aug 14 22:08:04 cblproxy1 kernel: fault virtual address = 0xa8 Aug 14 22:08:04 cblproxy1 kernel: fault code= supervisor write data, page not present Aug 14 22:08:04 cblproxy1 kernel: instruction pointer = 0x20:0x808b0562 Aug 14 22:08:04 cblproxy1 kernel: stack pointer = 0x28:0xff80002238f0 Aug 14 22:08:04 cblproxy1 kernel: frame pointer = 0x28:0xff8000223910 Aug 14 22:08:04 cblproxy1 kernel: code segment = base 0x0, limit 0xf, type 0x1b Aug 14 22:08:04 cblproxy1 kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Aug 14 22:08:04 cblproxy1 kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Aug 14 22:08:04 cblproxy1 kernel: current process = 1 (init) Aug 14 22:08:04 cblproxy1 kernel: trap number = 12 Aug 14 22:08:04 cblproxy1 kernel: panic: page fault Aug 14 22:08:04 cblproxy1 kernel: cpuid = 0 Aug 14 22:08:04 cblproxy1 kernel: KDB: stack backtrace: Aug 14 22:08:04 cblproxy1 kernel: #0 0x808ddaf0 at kdb_backtrace+0x60 Aug 14 22:08:04 cblproxy1 kernel: #1 0x808a951d at panic+0x1fd Aug 14 22:08:04 cblproxy1 kernel: #2 0x80b81578 at trap_fatal+0x388 Aug 14 22:08:04 cblproxy1 kernel: #3 0x80b81836 at trap_pfault+0x2a6 Aug 14 22:08:04 cblproxy1 kernel: #4 0x80b80ea1 at trap+0x2a1 Aug 14 22:08:04 cblproxy1 kernel: #5 0x80b6c7b3 at calltrap+0x8 Aug 14 22:08:04 cblproxy1 kernel: #6 0x815276da at zfsctl_umount_snapshots+0x8a Aug 14 22:08:04 cblproxy1 kernel: #7 0x81536766 at zfs_umount+0x76 Aug 14 22:08:04 cblproxy1 kernel: #8 0x809340bc at dounmount+0x3cc Aug 14 22:08:04 cblproxy1 kernel: #9 0x8093c101 at vfs_unmountall+0x71 Aug 14 22:08:04 cblproxy1 kernel: #10 0x808a8eae at kern_reboot+0x4ee Aug 14 22:08:04 cblproxy1 kernel: #11 0x808a89c0 at kern_reboot+0 Aug 14 22:08:04 cblproxy1 kernel: #12 0x80b81dab at amd64_syscall+0x29b Aug 14 22:08:04 cblproxy1 kernel: #13
Re: ZFS Snapshots Not able to be accessed under .zfs/snapshot/name
On 09/09/2013 22:38, dweimer wrote: A quick update on this, in case anyone else runs into it, I did finally try on the 2nd of this month to delete my UFS volume, and create a new ZFS volume to replace it. I recreated the Squid cache directories and let squid start over building up cache. So far their hasn't been a noticeable impact on performance with the switch over, and the snapshot problem has not reoccurred since making the change. Its only a week into running this way but the problem before started within 36-48 hours. From what you mentioned earlier you appear to use dates in your snapshot names. So kern/161968 - shouldn't affect you. For others using zfs volumes - check kern/161968 : [zfs] [hang] renaming snapshot with -r including a zvol snapshot causes total ZFS freeze/lockup ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS Snapshots Not able to be accessed under .zfs/snapshot/name
On 08/15/2013 10:00 am, dweimer wrote: On 08/14/2013 9:43 pm, Shane Ambler wrote: On 14/08/2013 22:57, dweimer wrote: I have a few systems running on ZFS with a backup script that creates snapshots, then backs up the .zfs/snapshot/name directory to make sure open files are not missed. This has been working great but all of the sudden one of my systems has stopped working. It takes the snapshots fine, zfs list -t spnapshot shows the snapshots, but if you do an ls command, on the .zfs/snapshot/ directory it returns not a directory. part of the zfs list output: NAMEUSED AVAIL REFER MOUNTPOINT zroot 4.48G 29.7G31K none zroot/ROOT 2.92G 29.7G31K none zroot/ROOT/91p5-20130812 2.92G 29.7G 2.92G legacy zroot/home 144K 29.7G 122K /home part of the zfs list -t snapshot output: NAMEUSED AVAIL REFER MOUNTPOINT zroot/ROOT/91p5-20130812@91p5-20130812--bsnap 340K - 2.92G - zroot/home@home--bsnap 22K - 122K - ls /.zfs/snapshot/91p5-20130812--bsnap/ Does work at the right now, since the last reboot, but wasn't always working, this is my boot environment. if I do ls /home/.zfs/snapshot/, result is: ls: /home/.zfs/snapshot/: Not a directory if I do ls /home/.zfs, result is: ls: snapshot: Bad file descriptor shares I have tried zpool scrub zroot, no errors were found, if I reboot the system I can get one good backup, then I start having problems. Anyone else ever ran into this, any suggestions as to a fix? System is running FreeBSD 9.1-RELEASE-p5 #1 r253764: Mon Jul 29 15:07:35 CDT 2013, zpool is running version 28, zfs is running version 5 I can say I've had this problem. Not certain what fixed it. I do remember I decided to stop snapshoting if I couldn't access them and deleted existing snapshots. I later restarted the machine before I went back for another look and they were working. So my guess is a restart without existing snapshots may be the key. Now if only we could find out what started the issue so we can stop it happening again. I had actually rebooted it last night, prior to seeing this message, I do know it didn't have any snapshots this time. As I am booting from ZFS using boot environments I may have had an older boot environment still on the system the last time it was rebooted. Backups ran great last night after the reboot, and I was able to kick off my pre-backup job and access all the snapshots today. Hopefully it doesn't come back, but if it does I will see if I can find anything else wrong. FYI, It didn't shutdown cleanly, so if this helps anyone find the issue, this is from my system logs: Aug 14 22:08:04 cblproxy1 kernel: Aug 14 22:08:04 cblproxy1 kernel: Fatal trap 12: page fault while in kernel mode Aug 14 22:08:04 cblproxy1 kernel: cpuid = 0; apic id = 00 Aug 14 22:08:04 cblproxy1 kernel: fault virtual address = 0xa8 Aug 14 22:08:04 cblproxy1 kernel: fault code= supervisor write data, page not present Aug 14 22:08:04 cblproxy1 kernel: instruction pointer = 0x20:0x808b0562 Aug 14 22:08:04 cblproxy1 kernel: stack pointer = 0x28:0xff80002238f0 Aug 14 22:08:04 cblproxy1 kernel: frame pointer = 0x28:0xff8000223910 Aug 14 22:08:04 cblproxy1 kernel: code segment = base 0x0, limit 0xf, type 0x1b Aug 14 22:08:04 cblproxy1 kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Aug 14 22:08:04 cblproxy1 kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Aug 14 22:08:04 cblproxy1 kernel: current process = 1 (init) Aug 14 22:08:04 cblproxy1 kernel: trap number = 12 Aug 14 22:08:04 cblproxy1 kernel: panic: page fault Aug 14 22:08:04 cblproxy1 kernel: cpuid = 0 Aug 14 22:08:04 cblproxy1 kernel: KDB: stack backtrace: Aug 14 22:08:04 cblproxy1 kernel: #0 0x808ddaf0 at kdb_backtrace+0x60 Aug 14 22:08:04 cblproxy1 kernel: #1 0x808a951d at panic+0x1fd Aug 14 22:08:04 cblproxy1 kernel: #2 0x80b81578 at trap_fatal+0x388 Aug 14 22:08:04 cblproxy1 kernel: #3 0x80b81836 at trap_pfault+0x2a6 Aug 14 22:08:04 cblproxy1 kernel: #4 0x80b80ea1 at trap+0x2a1 Aug 14 22:08:04 cblproxy1 kernel: #5 0x80b6c7b3 at calltrap+0x8 Aug 14 22:08:04 cblproxy1 kernel: #6 0x815276da at zfsctl_umount_snapshots+0x8a Aug 14 22:08:04 cblproxy1 kernel: #7 0x81536766 at zfs_umount+0x76 Aug 14 22:08:04 cblproxy1 kernel: #8 0x809340bc at dounmount+0x3cc Aug 14 22:08:04 cblproxy1 kernel: #9 0x8093c101 at vfs_unmountall+0x71 Aug 14 22:08:04 cblproxy1 kernel: #10 0x808a8eae at kern_reboot+0x4ee Aug 14 22:08:04 cblproxy1 kernel: #11 0x808a89c0 at kern_reboot+0 Aug 14 22:08:04 cblproxy1 kernel: #12 0x80b81dab at amd64_syscall+0x29b Aug 14 22:08:04 cblproxy1 kernel: #13 0x80b6ca9b at Xfast_syscall+0xfb Well its
Re: ZFS Snapshots Not able to be accessed under .zfs/snapshot/name
On 08/14/2013 9:43 pm, Shane Ambler wrote: On 14/08/2013 22:57, dweimer wrote: I have a few systems running on ZFS with a backup script that creates snapshots, then backs up the .zfs/snapshot/name directory to make sure open files are not missed. This has been working great but all of the sudden one of my systems has stopped working. It takes the snapshots fine, zfs list -t spnapshot shows the snapshots, but if you do an ls command, on the .zfs/snapshot/ directory it returns not a directory. part of the zfs list output: NAMEUSED AVAIL REFER MOUNTPOINT zroot 4.48G 29.7G31K none zroot/ROOT 2.92G 29.7G31K none zroot/ROOT/91p5-20130812 2.92G 29.7G 2.92G legacy zroot/home 144K 29.7G 122K /home part of the zfs list -t snapshot output: NAMEUSED AVAIL REFER MOUNTPOINT zroot/ROOT/91p5-20130812@91p5-20130812--bsnap 340K - 2.92G - zroot/home@home--bsnap 22K - 122K - ls /.zfs/snapshot/91p5-20130812--bsnap/ Does work at the right now, since the last reboot, but wasn't always working, this is my boot environment. if I do ls /home/.zfs/snapshot/, result is: ls: /home/.zfs/snapshot/: Not a directory if I do ls /home/.zfs, result is: ls: snapshot: Bad file descriptor shares I have tried zpool scrub zroot, no errors were found, if I reboot the system I can get one good backup, then I start having problems. Anyone else ever ran into this, any suggestions as to a fix? System is running FreeBSD 9.1-RELEASE-p5 #1 r253764: Mon Jul 29 15:07:35 CDT 2013, zpool is running version 28, zfs is running version 5 I can say I've had this problem. Not certain what fixed it. I do remember I decided to stop snapshoting if I couldn't access them and deleted existing snapshots. I later restarted the machine before I went back for another look and they were working. So my guess is a restart without existing snapshots may be the key. Now if only we could find out what started the issue so we can stop it happening again. I had actually rebooted it last night, prior to seeing this message, I do know it didn't have any snapshots this time. As I am booting from ZFS using boot environments I may have had an older boot environment still on the system the last time it was rebooted. Backups ran great last night after the reboot, and I was able to kick off my pre-backup job and access all the snapshots today. Hopefully it doesn't come back, but if it does I will see if I can find anything else wrong. FYI, It didn't shutdown cleanly, so if this helps anyone find the issue, this is from my system logs: Aug 14 22:08:04 cblproxy1 kernel: Aug 14 22:08:04 cblproxy1 kernel: Fatal trap 12: page fault while in kernel mode Aug 14 22:08:04 cblproxy1 kernel: cpuid = 0; apic id = 00 Aug 14 22:08:04 cblproxy1 kernel: fault virtual address = 0xa8 Aug 14 22:08:04 cblproxy1 kernel: fault code= supervisor write data, page not present Aug 14 22:08:04 cblproxy1 kernel: instruction pointer = 0x20:0x808b0562 Aug 14 22:08:04 cblproxy1 kernel: stack pointer = 0x28:0xff80002238f0 Aug 14 22:08:04 cblproxy1 kernel: frame pointer = 0x28:0xff8000223910 Aug 14 22:08:04 cblproxy1 kernel: code segment = base 0x0, limit 0xf, type 0x1b Aug 14 22:08:04 cblproxy1 kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Aug 14 22:08:04 cblproxy1 kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Aug 14 22:08:04 cblproxy1 kernel: current process = 1 (init) Aug 14 22:08:04 cblproxy1 kernel: trap number = 12 Aug 14 22:08:04 cblproxy1 kernel: panic: page fault Aug 14 22:08:04 cblproxy1 kernel: cpuid = 0 Aug 14 22:08:04 cblproxy1 kernel: KDB: stack backtrace: Aug 14 22:08:04 cblproxy1 kernel: #0 0x808ddaf0 at kdb_backtrace+0x60 Aug 14 22:08:04 cblproxy1 kernel: #1 0x808a951d at panic+0x1fd Aug 14 22:08:04 cblproxy1 kernel: #2 0x80b81578 at trap_fatal+0x388 Aug 14 22:08:04 cblproxy1 kernel: #3 0x80b81836 at trap_pfault+0x2a6 Aug 14 22:08:04 cblproxy1 kernel: #4 0x80b80ea1 at trap+0x2a1 Aug 14 22:08:04 cblproxy1 kernel: #5 0x80b6c7b3 at calltrap+0x8 Aug 14 22:08:04 cblproxy1 kernel: #6 0x815276da at zfsctl_umount_snapshots+0x8a Aug 14 22:08:04 cblproxy1 kernel: #7 0x81536766 at zfs_umount+0x76 Aug 14 22:08:04 cblproxy1 kernel: #8 0x809340bc at dounmount+0x3cc Aug 14 22:08:04 cblproxy1 kernel: #9 0x8093c101 at vfs_unmountall+0x71 Aug 14 22:08:04 cblproxy1 kernel: #10 0x808a8eae at kern_reboot+0x4ee Aug 14 22:08:04 cblproxy1 kernel: #11 0x808a89c0 at kern_reboot+0 Aug 14 22:08:04 cblproxy1 kernel: #12 0x80b81dab at amd64_syscall+0x29b Aug 14 22:08:04 cblproxy1 kernel: #13 0x80b6ca9b at Xfast_syscall+0xfb -- Thanks, Dean E. Weimer http
ZFS Snapshots Not able to be accessed under .zfs/snapshot/name
I have a few systems running on ZFS with a backup script that creates snapshots, then backs up the .zfs/snapshot/name directory to make sure open files are not missed. This has been working great but all of the sudden one of my systems has stopped working. It takes the snapshots fine, zfs list -t spnapshot shows the snapshots, but if you do an ls command, on the .zfs/snapshot/ directory it returns not a directory. part of the zfs list output: NAMEUSED AVAIL REFER MOUNTPOINT zroot 4.48G 29.7G31K none zroot/ROOT 2.92G 29.7G31K none zroot/ROOT/91p5-20130812 2.92G 29.7G 2.92G legacy zroot/home 144K 29.7G 122K /home part of the zfs list -t snapshot output: NAMEUSED AVAIL REFER MOUNTPOINT zroot/ROOT/91p5-20130812@91p5-20130812--bsnap 340K - 2.92G - zroot/home@home--bsnap 22K - 122K - ls /.zfs/snapshot/91p5-20130812--bsnap/ Does work at the right now, since the last reboot, but wasn't always working, this is my boot environment. if I do ls /home/.zfs/snapshot/, result is: ls: /home/.zfs/snapshot/: Not a directory if I do ls /home/.zfs, result is: ls: snapshot: Bad file descriptor shares I have tried zpool scrub zroot, no errors were found, if I reboot the system I can get one good backup, then I start having problems. Anyone else ever ran into this, any suggestions as to a fix? System is running FreeBSD 9.1-RELEASE-p5 #1 r253764: Mon Jul 29 15:07:35 CDT 2013, zpool is running version 28, zfs is running version 5 -- Thanks, Dean E. Weimer http://www.dweimer.net/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS Snapshots Not able to be accessed under .zfs/snapshot/name
On 14/08/2013 22:57, dweimer wrote: I have a few systems running on ZFS with a backup script that creates snapshots, then backs up the .zfs/snapshot/name directory to make sure open files are not missed. This has been working great but all of the sudden one of my systems has stopped working. It takes the snapshots fine, zfs list -t spnapshot shows the snapshots, but if you do an ls command, on the .zfs/snapshot/ directory it returns not a directory. part of the zfs list output: NAMEUSED AVAIL REFER MOUNTPOINT zroot 4.48G 29.7G31K none zroot/ROOT 2.92G 29.7G31K none zroot/ROOT/91p5-20130812 2.92G 29.7G 2.92G legacy zroot/home 144K 29.7G 122K /home part of the zfs list -t snapshot output: NAMEUSED AVAIL REFER MOUNTPOINT zroot/ROOT/91p5-20130812@91p5-20130812--bsnap 340K - 2.92G - zroot/home@home--bsnap 22K - 122K - ls /.zfs/snapshot/91p5-20130812--bsnap/ Does work at the right now, since the last reboot, but wasn't always working, this is my boot environment. if I do ls /home/.zfs/snapshot/, result is: ls: /home/.zfs/snapshot/: Not a directory if I do ls /home/.zfs, result is: ls: snapshot: Bad file descriptor shares I have tried zpool scrub zroot, no errors were found, if I reboot the system I can get one good backup, then I start having problems. Anyone else ever ran into this, any suggestions as to a fix? System is running FreeBSD 9.1-RELEASE-p5 #1 r253764: Mon Jul 29 15:07:35 CDT 2013, zpool is running version 28, zfs is running version 5 I can say I've had this problem. Not certain what fixed it. I do remember I decided to stop snapshoting if I couldn't access them and deleted existing snapshots. I later restarted the machine before I went back for another look and they were working. So my guess is a restart without existing snapshots may be the key. Now if only we could find out what started the issue so we can stop it happening again. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD 9.0 Snapshots
Has been fixed, but you need to understand release engineering better than me if you want the fix now. Fixed? What did you mean? SUJ and snapshots still doesn't work together. -- View this message in context: http://freebsd.1045724.n5.nabble.com/FreeBSD-9-0-Snapshots-tp5585872p5589250.html Sent from the freebsd-questions mailing list archive at Nabble.com. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD 9.0 Snapshots
See PR http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/161674 (This bug was fixed in head in -r230249, by mckusick Feb 7). To see the fix in your system now, I think you could update your system from the CURRENT branch (but as I said, you'll need to understand release engineering better than I). For me, I'm following RELEASE and I'm just going to wait until it appears. Dale - Original Message - From: timp tim...@gmail.com To: freebsd-questions@freebsd.org Sent: Friday, March 23, 2012 6:03:37 AM Subject: Re: FreeBSD 9.0 Snapshots Has been fixed, but you need to understand release engineering better than me if you want the fix now. Fixed? What did you mean? SUJ and snapshots still doesn't work together. -- View this message in context: http://freebsd.1045724.n5.nabble.com/FreeBSD-9-0-Snapshots-tp5585872p5589250.html Sent from the freebsd-questions mailing list archive at Nabble.com. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD 9.0 Snapshots
I saw it and I don't understand why Kirk wrote 'fixed' because latest CURRENT prints warning message like 'SUJ doesn't work with snapshots' 23.03.2012 19:05 пользователь Dale Scott dalesc...@shaw.ca написал: See PR http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/161674 (This bug was fixed in head in -r230249, by mckusick Feb 7). To see the fix in your system now, I think you could update your system from the CURRENT branch (but as I said, you'll need to understand release engineering better than I). For me, I'm following RELEASE and I'm just going to wait until it appears. Dale - Original Message - From: timp tim...@gmail.com To: freebsd-questions@freebsd.org Sent: Friday, March 23, 2012 6:03:37 AM Subject: Re: FreeBSD 9.0 Snapshots Has been fixed, but you need to understand release engineering better than me if you want the fix now. Fixed? What did you mean? SUJ and snapshots still doesn't work together. -- View this message in context: http://freebsd.1045724.n5.nabble.com/FreeBSD-9-0-Snapshots-tp5585872p5589250.html Sent from the freebsd-questions mailing list archive at Nabble.com. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD 9.0 Snapshots
On Fri, Mar 23, 2012 at 1:16 PM, Pavel Timofeev tim...@gmail.com wrote: I saw it and I don't understand why Kirk wrote 'fixed' because latest CURRENT prints warning message like 'SUJ doesn't work with snapshots' It could be the warning message had been overlooked or indeed there is reason for it to still remain. Either way, you'll probably have better luck asking on freebsd-fs@ -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
FreeBSD 9.0 Snapshots
Has anyone been using Snapshots on UFS with journaled soft-updates enabled? I have a couple of new systems built after 9.0 came out, my backup scripts take snapshots, and then mount them to backup the files, the couple older servers that I upgraded from 8.2 to 9.0 from source are not having any problems, a quick check shows their file systems aren't running the new journaled Soft-Updates options. The new systems which are, frequently hang up and become unresponsive when taking the snapshots. -- Thanks, Dean E. Weimer http://www.dweimer.net/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD 9.0 Snapshots
Dean E. Weimer wrote: Has anyone been using Snapshots on UFS with journaled soft-updates enabled? I have a couple of new systems built after 9.0 came out, my backup scripts take snapshots, and then mount them to backup the files, the couple older servers that I upgraded from 8.2 to 9.0 from source are not having any problems, a quick check shows their file systems aren't running the new journaled Soft-Updates options. The new systems which are, frequently hang up and become unresponsive when taking the snapshots. Known issue, don't use SUJ with snapshots on 9.0 for now. -- Sphinx of black quartz judge my vow. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD 9.0 Snapshots
Has been fixed, but you need to understand release engineering better than me if you want the fix now. -- There are no passengers on spaceship earth. We are all crew. - Original Message - From: Volodymyr Kostyrko c.kw...@gmail.com To: dwei...@dweimer.net Cc: freebsd-questions@freebsd.org Sent: Thursday, March 22, 2012 9:28:25 AM Subject: Re: FreeBSD 9.0 Snapshots Dean E. Weimer wrote: Has anyone been using Snapshots on UFS with journaled soft-updates enabled? I have a couple of new systems built after 9.0 came out, my backup scripts take snapshots, and then mount them to backup the files, the couple older servers that I upgraded from 8.2 to 9.0 from source are not having any problems, a quick check shows their file systems aren't running the new journaled Soft-Updates options. The new systems which are, frequently hang up and become unresponsive when taking the snapshots. Known issue, don't use SUJ with snapshots on 9.0 for now. -- Sphinx of black quartz judge my vow. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Help Finding ZFS snapshots
On 5 September 2011 16:58, Gene f...@brightstar.bomgardner.net wrote: On Mon, 05 Sep 2011 11:35:34 -0400, Daniel Staal wrote --As of September 5, 2011 10:23:32 AM -0500, Gene is alleged to have said: On Mon, 05 Sep 2011 10:48:22 -0400, Daniel Staal wrote --As of September 5, 2011 8:13:52 AM -0500, Gene is alleged to have said: Using FreeBSD 8.1, amd64 - I wanted to recover files from a snapshot of usr/home. Everything I've found via googling refers to a link such as path/zfs/.snapshot --As for the rest, it is mine. Try path/.zfs. ;) (Which, on my system, then has a 'snapshot' directory, which holds all the snapshots.) Daniel T. Staal No such luck. The following: cd / ls -R | grep -i zfs finds only 'zfs' directories in the source tree and ports. Other ideas? I know the snapshots exist, I can see 'em with zfs list -t snapshot. --As for the rest, it is mine. Don't check if the directory is there first. It isn't. Just 'cd' to it, and it will exist. Daniel T. Staal Well I'll be hornswaggled ... Thanks! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org as others have posted its hidden. This is for good reason though. Just imagine you backup program trawling your 10 TB array that has 100 historical snapshots. Suddenly you are backing up 1 PB 8( ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Help Finding ZFS snapshots
Hi All: Using FreeBSD 8.1, amd64 - I wanted to recover files from a snapshot of usr/home. Everything I've found via googling refers to a link such as path/zfs/.snapshot However, I'll be darned if I can find any such link. Snapshots existing are: NAME USED AVAIL REFER MOUNTPOINT zroot/usr@2011-01-02 1.06G - 10.2G - zroot/usr/home@2011-07-09 419M - 11.9G - zroot/usr/ports@2011-01-02 720M - 723M - zroot/usr/ports/distfiles@2011-01-02 310K - 2.72G - zroot/usr/ports/packages@2011-01-02 20.0K - 24.0K - zroot/usr/src@2011-01-02 159M - 310M - And ZFS is version 3: This system is currently running ZFS filesystem version 3. All filesystems are formatted with the current version. === Either 1) I'm looking in all the wrong places, 2) the link was somehow deleted, 3) ZFS has changed TWTAD (the way things are done). Can anyone point me in the right direction? Thanks. -- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Help Finding ZFS snapshots
On 05/09/2011 14:13, Gene wrote: Either 1) I'm looking in all the wrong places, 2) the link was somehow deleted, 3) ZFS has changed TWTAD (the way things are done). Can anyone point me in the right direction? ZFS snapshots automount themselves when you cd to the snapshot directory. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matt...@infracaninophile.co.uk Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: Help Finding ZFS snapshots
On Mon, 05 Sep 2011 14:22:45 +0100, Matthew Seaman wrote On 05/09/2011 14:13, Gene wrote: Either 1) I'm looking in all the wrong places, 2) the link was somehow deleted, 3) ZFS has changed TWTAD (the way things are done). Can anyone point me in the right direction? ZFS snapshots automount themselves when you cd to the snapshot directory. Cheers, Matthew The problem is finding the snapshot directory to cd into... -- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Help Finding ZFS snapshots
On Mon, 05 Sep 2011 10:48:22 -0400, Daniel Staal wrote --As of September 5, 2011 8:13:52 AM -0500, Gene is alleged to have said: Using FreeBSD 8.1, amd64 - I wanted to recover files from a snapshot of usr/home. Everything I've found via googling refers to a link such as path/zfs/.snapshot --As for the rest, it is mine. Try path/.zfs. ;) (Which, on my system, then has a 'snapshot' directory, which holds all the snapshots.) Daniel T. Staal No such luck. The following: cd / ls -R | grep -i zfs finds only 'zfs' directories in the source tree and ports. Other ideas? I know the snapshots exist, I can see 'em with zfs list -t snapshot. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Help Finding ZFS snapshots
--As of September 5, 2011 10:23:32 AM -0500, Gene is alleged to have said: On Mon, 05 Sep 2011 10:48:22 -0400, Daniel Staal wrote --As of September 5, 2011 8:13:52 AM -0500, Gene is alleged to have said: Using FreeBSD 8.1, amd64 - I wanted to recover files from a snapshot of usr/home. Everything I've found via googling refers to a link such as path/zfs/.snapshot --As for the rest, it is mine. Try path/.zfs. ;) (Which, on my system, then has a 'snapshot' directory, which holds all the snapshots.) Daniel T. Staal No such luck. The following: cd / ls -R | grep -i zfs finds only 'zfs' directories in the source tree and ports. Other ideas? I know the snapshots exist, I can see 'em with zfs list -t snapshot. --As for the rest, it is mine. Don't check if the directory is there first. It isn't. Just 'cd' to it, and it will exist. Daniel T. Staal --- This email copyright the author. Unless otherwise noted, you are expressly allowed to retransmit, quote, or otherwise use the contents for non-commercial purposes. This copyright will expire 5 years after the author's death, or in 30 years, whichever is longer, unless such a period is in excess of local copyright law. --- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Help Finding ZFS snapshots
On Mon, 05 Sep 2011 11:35:34 -0400, Daniel Staal wrote --As of September 5, 2011 10:23:32 AM -0500, Gene is alleged to have said: On Mon, 05 Sep 2011 10:48:22 -0400, Daniel Staal wrote --As of September 5, 2011 8:13:52 AM -0500, Gene is alleged to have said: Using FreeBSD 8.1, amd64 - I wanted to recover files from a snapshot of usr/home. Everything I've found via googling refers to a link such as path/zfs/.snapshot --As for the rest, it is mine. Try path/.zfs. ;) (Which, on my system, then has a 'snapshot' directory, which holds all the snapshots.) Daniel T. Staal No such luck. The following: cd / ls -R | grep -i zfs finds only 'zfs' directories in the source tree and ports. Other ideas? I know the snapshots exist, I can see 'em with zfs list -t snapshot. --As for the rest, it is mine. Don't check if the directory is there first. It isn't. Just 'cd' to it, and it will exist. Daniel T. Staal Well I'll be hornswaggled ... Thanks! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Help Finding ZFS snapshots
2011-09-05 17:23, Gene skrev: On Mon, 05 Sep 2011 10:48:22 -0400, Daniel Staal wrote --As of September 5, 2011 8:13:52 AM -0500, Gene is alleged to have said: Using FreeBSD 8.1, amd64 - I wanted to recover files from a snapshot of usr/home. Everything I've found via googling refers to a link such as path/zfs/.snapshot --As for the rest, it is mine. Try path/.zfs. ;) (Which, on my system, then has a 'snapshot' directory, which holds all the snapshots.) Daniel T. Staal No such luck. The following: cd / ls -R | grep -i zfs finds only 'zfs' directories in the source tree and ports. Other ideas? I know the snapshots exist, I can see 'em with zfs list -t snapshot. The .zfs directory is normally hidden, so it won't even show up on ls -a output. You have to either explicitly cd to it or make it visible by zfs set snapdir=visible filesystem ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Help Finding ZFS snapshots
Gene f...@brightstar.bomgardner.net writes: On Mon, 05 Sep 2011 10:48:22 -0400, Daniel Staal wrote --As of September 5, 2011 8:13:52 AM -0500, Gene is alleged to have said: Using FreeBSD 8.1, amd64 - I wanted to recover files from a snapshot of usr/home. Everything I've found via googling refers to a link such as path/zfs/.snapshot --As for the rest, it is mine. Try path/.zfs. ;) (Which, on my system, then has a 'snapshot' directory, which holds all the snapshots.) Daniel T. Staal No such luck. The following: cd / ls -R | grep -i zfs finds only 'zfs' directories in the source tree and ports. Other ideas? I know the snapshots exist, I can see 'em with zfs list -t snapshot. The .zfs directory is hidden by default so you have to specifically ls or go into them. Do a 'ls' on the base directory of any zfs file system, and then add .zfs to the end and you should see the .snapshots directory. -- Carl Johnsonca...@peak.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
daily snapshots updated (Was: Re: Where to download latest FreeBSD snapshots)
Test Rat ttse...@gmail.com wrote in 86y5z1ymyi@gmail.com: tt Can you add architecture name to HEAD snapshots? It often saves time tt checking whether snapshot is suitable for testing months after being tt dowloaded. Thank you for your feedback. While I have received various ideas and am still working on them, I recently added changes for the followng: 1. Use $TARGET and $TARGET_ARCH in ISO image names. Now it is like the following: FreeBSD-9.0-HEAD-20110809-JPSNAP-i386-i386-bootonly.iso Also, SHA-256 checksum files have been added in the same directory. 2. Use a clean build environment. This should fix an iso9660 format breakage issue in makefs(8). 3. The uncompressed tree of the release tarballs is added under the trees/ directory. Currently, snapshots of 7 platforms are being built natively. -- Hiroki pgpK9EK7BYOTr.pgp Description: PGP signature
Re: Where to download latest FreeBSD snapshots
On Mon, Jun 27, 2011 at 6:54 PM, Hiroki Sato h...@freebsd.org wrote: Pan Tsu iny...@gmail.com wrote in 864o3dtsey@gmail.com: in Hiroki Sato h...@freebsd.org writes: in in Hello, in in dave jones s.dave.jo...@gmail.com wrote in in BANLkTikR-GL9LFkTL6f=pm5vcazaftk...@mail.gmail.com: in in s. It seems that allbsd.org is up, but I can't find the HEAD snapshots, in s. only RELENG. in s. Would you like to build HEAD snapshots? Thank you very much. in in Building snapshots of HEAD and RELENG_[67] are temporarily disabled in because a maintenance work is now in progress. They will be back on in the page in the next week. in in Are there more places for *daily* HEAD snapshots? I used them a few in times to report regressions with a clean environment. The HEAD snapshot build is finally getting recovered (currently for amd64 and i386 only, though). Some hardware failure prevented the build cluster from working. -- Hiroki Thank you so much for providing this service Hiroki! -Brandon ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Where to download latest FreeBSD snapshots
Hiroki Sato h...@freebsd.org writes: Pan Tsu iny...@gmail.com wrote in 864o3dtsey@gmail.com: in Hiroki Sato h...@freebsd.org writes: in in Hello, in in dave jones s.dave.jo...@gmail.com wrote inin BANLkTikR-GL9LFkTL6f=pm5vcazaftk...@mail.gmail.com: in in s. It seems that allbsd.org is up, but I can't find the HEAD snapshots, in s. only RELENG. in s. Would you like to build HEAD snapshots? Thank you very much. in in Building snapshots of HEAD and RELENG_[67] are temporarily disabled in because a maintenance work is now in progress. They will be back on in the page in the next week. in in Are there more places for *daily* HEAD snapshots? I used them a few in times to report regressions with a clean environment. The HEAD snapshot build is finally getting recovered (currently for amd64 and i386 only, though). Some hardware failure prevented the build cluster from working. Can you add architecture name to HEAD snapshots? It often saves time checking whether snapshot is suitable for testing months after being dowloaded. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: MC and snapshots
On Fri, July 29, 2011 2:26 pm, Dick Hoogendijk wrote: Can one of you tell me why it is not possible to browse .zfs directories (from snapshots) with midnight commander? I'm running FreeBSD-8.2 w/ mc from ports. Manually switching to .zfs and it's subdirectories does show the snapshotted files, but I would like to be able to browse them (its so much easier). I know that the snapshots aren't mounted by default. (And are mounted/unmounted on the fly if needed.) That's probably part of it, at least. Have you tried setting the 'snapdir' property to 'visible'? I don't know that would help, but it'd be worth trying. Daniel T. Staal --- This email copyright the author. Unless otherwise noted, you are expressly allowed to retransmit, quote, or otherwise use the contents for non-commercial purposes. This copyright will expire 5 years after the author's death, or in 30 years, whichever is longer, unless such a period is in excess of local copyright law. --- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: MC and snapshots
Op 29-7-2011 22:14, Daniel Staal schreef: On Fri, July 29, 2011 2:26 pm, Dick Hoogendijk wrote: Can one of you tell me why it is not possible to browse .zfs directories (from snapshots) with midnight commander? I'm running FreeBSD-8.2 w/ mc from ports. Manually switching to .zfs and it's subdirectories does show the snapshotted files, but I would like to be able to browse them (its so much easier). I know that the snapshots aren't mounted by default. (And are mounted/unmounted on the fly if needed.) That's probably part of it, at least. Have you tried setting the 'snapdir' property to 'visible'? I don't know that would help, but it'd be worth trying. I *can* go to the snapshots and see the files. I even can copy one or more of them to the live system. However, the moment I try to do this from within Midnight Commander I'm kicked out of the snapshot directory right into the root of the live filesystem. That way not able to manipulated snapshot files. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Where to download latest FreeBSD snapshots
Pan Tsu iny...@gmail.com wrote in 864o3dtsey@gmail.com: in Hiroki Sato h...@freebsd.org writes: in in Hello, in in dave jones s.dave.jo...@gmail.com wrote inin BANLkTikR-GL9LFkTL6f=pm5vcazaftk...@mail.gmail.com: in in s. It seems that allbsd.org is up, but I can't find the HEAD snapshots, in s. only RELENG. in s. Would you like to build HEAD snapshots? Thank you very much. in in Building snapshots of HEAD and RELENG_[67] are temporarily disabled in because a maintenance work is now in progress. They will be back on in the page in the next week. in in Are there more places for *daily* HEAD snapshots? I used them a few in times to report regressions with a clean environment. The HEAD snapshot build is finally getting recovered (currently for amd64 and i386 only, though). Some hardware failure prevented the build cluster from working. -- Hiroki pgpJPL7F3aVxA.pgp Description: PGP signature
Re: Where to download latest FreeBSD snapshots
Hiroki Sato h...@freebsd.org writes: Hello, dave jones s.dave.jo...@gmail.com wrote in BANLkTikR-GL9LFkTL6f=pm5vcazaftk...@mail.gmail.com: s. It seems that allbsd.org is up, but I can't find the HEAD snapshots, s. only RELENG. s. Would you like to build HEAD snapshots? Thank you very much. Building snapshots of HEAD and RELENG_[67] are temporarily disabled because a maintenance work is now in progress. They will be back on the page in the next week. Are there more places for *daily* HEAD snapshots? I used them a few times to report regressions with a clean environment. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Where to download latest FreeBSD snapshots
Hello, dave jones s.dave.jo...@gmail.com wrote in BANLkTikR-GL9LFkTL6f=pm5vcazaftk...@mail.gmail.com: s. It seems that allbsd.org is up, but I can't find the HEAD snapshots, s. only RELENG. s. Would you like to build HEAD snapshots? Thank you very much. Building snapshots of HEAD and RELENG_[67] are temporarily disabled because a maintenance work is now in progress. They will be back on the page in the next week. -- Hiroki pgpjlSeTA8td1.pgp Description: PGP signature
Where to download latest FreeBSD snapshots
Hello, It seems that www.allbsd.org is down and ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/201102 is old. Anyone knows where I can get the FreeBSD snapshots? Thank you. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
UFS Snapshots and iowait
I have started using mount -u -o snapshot as part of my backup process in order to have a week worth of local differential backups to allow quick and easy recovery of lost/overwritten/etc files. The snapshot of the partition (~250G and 2.3 million inodes used. ~10GB of data change per day) takes around 10 minutes to complete. During the first 5 minutes everything seems to be find, but during the second 5 minutes the Apache processes that are logging to this drive start building up in L (logging) state until they hit MaxClients. Is this just due to the very high io bandwidth usage associated with making a snapshot, or does the creation of this snapshot completely block IO writes for around 5 minutes? Any suggested workarounds? I already bumped up the number of Apache slots to 166% but it looks like I would have to increase the number much more to use that as a primary solution. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Where's Snapshots and pub.allbsd.org
Aloha, Anybody on the FreeBSD list know what has happened to the snapshots that have not been available on FreeBSD.org since last Sept.? Also what happened to pub.allbsd.org snapshots? Are there new URL's for current and daily snapshots to test? ~Al Plant - Honolulu, Hawaii - Phone: 808-284-2740 + http://hawaiidakine.com + http://freebsdinfo.org + + http://aloha50.net - Supporting - FreeBSD 7.2 - 8.0 - 9* + email: n...@hdk5.net All that's really worth doing is what we do for others.- Lewis Carrol ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: July snapshots
On Mon, 13 Jul 2009 08:31:44 -0500 Andrew Gould andrewlylego...@gmail.com wrote: Does anyone know if any snapshots (iso files at ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/) will created in July? I don't know, but you can always find daily snapshots at http://pub.allbsd.org/FreeBSD-snapshots/ -- Bruce Cran ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: July snapshots
On Wed, Jul 15, 2009 at 1:52 AM, Bruce Cranbr...@cran.org.uk wrote: On Mon, 13 Jul 2009 08:31:44 -0500 Andrew Gould andrewlylego...@gmail.com wrote: Does anyone know if any snapshots (iso files at ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/) will created in July? I don't know, but you can always find daily snapshots at http://pub.allbsd.org/FreeBSD-snapshots/ -- Bruce Cran Thanks, You just saved me a lot of compile time. Andrew ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
July snapshots
Does anyone know if any snapshots (iso files at ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/) will created in July? Thanks, Andrew ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Snapshots
At 04:00 AM 5/5/2009, Johan Hendriks wrote: Are there no more snapshots of current? The last is from 02-2009 Regards, Johan I downloaded one from this month a couple days ago. You should see a May snapshot available. -Derek -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Snapshots
On 8/5/09 12:36, Derek Ragona wrote: At 04:00 AM 5/5/2009, Johan Hendriks wrote: Are there no more snapshots of current? The last is from 02-2009 Regards, Johan I downloaded one from this month a couple days ago. You should see a May snapshot available. -Derek Also see http://pub.allbsd.org/FreeBSD-snapshots/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Snapshots
Are there no more snapshots of current? The last is from 02-2009 Regards, Johan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
snapshots
I was looking into snapshots using the doc below to try and see if delta copies with rsync matched data better if I take a snapshot of a file system, mount it and rsync transfer the data. Right now, dumps don't seem to match much data due to changes in the sequences and a very active mail cache pgsql db. http://people.freebsd.org/~rse/snapshot/ I created the snapshot and got it mounted, took a minute or so for each to do this on my /usr ufs file system on FreeBSD 6.2 with 2GB RAM, dual P4 procs with SMP and RAID 5 (I know, will be changing soon) SATA150 drives. In the midst of doing this, the currently running pgsql db, remember I said it was very active amavisd-maia supporting two mail gateways as a mail cache if you know Maia Mailguard, I started getting these messages and things stopped responding well... Sep 24 16:59:02 mx1 kernel: aac0: COMMAND 0xc6530fc0 TIMEOUT AFTER 38 SECONDS Did a reboot, it tooks several minutes to start reboot after the command. I even got some of these kernel messages after the reboot, not until I rm -f the snapshot file did it stop. I am wondering how I need to handle snapshots when trying this, especially if included in a routine backup script? I haven't made it to the solution part of the above doc yet, wanted to check here before I try anything again. Also, haven't tested my rsync until tonight to see if the file system transfer will make a difference in my ability to match data better. But I'd still like use the snapshot as a backup solution. -- Robert ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Snapshots fail on busy filesystem
Greetings to all, I'm trying to set up an automated system wherein snapshots of the file system are taken prior to a backup run. The problem I'm running into can best be described by the following bug report: http://www.freebsd.org/cgi/query-pr.cgi?pr=100365cat= I get this same problem completely at random, and thus is most likely caused by the busy file system, I just don't have anything in place to prove it as the bug submitter does. Has this bug ever been *really* fixed? The bug report seems to try to call it fixed. I even contacted the bug reporter about this and he admitted that the bug had never been fixed, but that he'd moved on and never found a way around this. If the bug has never been fixed, has someone developed a work around? My primary goal is to make sure that I have a good backup of the databases running on our servers when we run a backup with Bacula. If there is a method other than snapshots to make absolutely sure that my databases are backed up correctly, I would consider that as well. Please help. My backup system is running perfectly except for this nagging sensation that my databases aren't being backed up correctly. Any assistance is appreciated. Thank you, Chris Morris -- S i x F e e t U p | Nowhere to go but open source Silicon Valley: +1 (650) 401-8579 x609 Midwest: +1 (317) 861-5948 x609 Toll-Free: 1-866-SIX-FEET mailto:[EMAIL PROTECTED] http://www.sixfeetup.com | Zope/Plone Custom Development ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Snapshots fail on busy filesystem
On Wed, Aug 22, 2007 at 10:01:35AM -0400, Chris Morris wrote: Greetings to all, I'm trying to set up an automated system wherein snapshots of the file system are taken prior to a backup run. The problem I'm running into can best be described by the following bug report: http://www.freebsd.org/cgi/query-pr.cgi?pr=100365cat= I get this same problem completely at random, and thus is most likely caused by the busy file system, I just don't have anything in place to prove it as the bug submitter does. Has this bug ever been *really* fixed? The bug report seems to try to call it fixed. I even contacted the bug reporter about this and he admitted that the bug had never been fixed, but that he'd moved on and never found a way around this. Sounds like you both need to talk to [EMAIL PROTECTED] Of course as long as he thinks the problem is fixed he is not going to work on fixing it :) Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Amanda and FreeBSD snapshots
I am using an amanda 2.5 server on a FreeBSD 6.2 host. I have some FreeBSD 5.3 clients running the amanda 2.4.4 client side version. I also have an old BSD/OS machine running some flavor of amanda client. I noticed that the dumps use the ushf options. BUt it would seem to me that you'd also want the -L option as mentioned on the FreeBSD machine's dump command man page: -L This option is to notify dump that it is dumping a live file system. To obtain a consistent dump image, dump takes a snapshot of the file system in the .snap directory in the root of the file system being dumped and then does a dump of the snapshot. The snapshot is unlinked as soon as the dump starts, and is thus removed when the dump is complete. This option is ignored for unmounted or read-only file systems. If the .snap directory does not exist in the root of the file system being dumped, a warning will be issued and the dump will revert to the standard behavior. This problem can be corrected by creating a .snap directory in the root of the file system to be dumped; its owner should be ``root'', its group should be ``operator'', and its mode should be ``0770''. This seems like a good option to use when running backups on the FreeBSD clients. Any amanda users out there know how to get amanda to use the -L option on the FreeBSD machines, but not on the BSD/OS machine, suince it can't do snapshots ? Thanks Chris Kottaridis([EMAIL PROTECTED]) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: fsck, soft-updates, and snapshots
On Tuesday 13 March 2007 05:52, David Cecil wrote: Can anyone tell me src/sys/ufs/ffs/README.softupdates and README.snapshots are up to date? The former is dated June 2000, so it's almost 7 years old. The snapshots readme is only 2 years younger and mentions the code being alpha-test. I can answer indirectly. sysinstall, the default FreeBSD installer, enables soft-updates by default on every filesystem, except the root filesystem. I think this is true since the 4.x days. This must make them pretty well tested. I do not the exact status of snapshots. Perhaps you should ask freebsd-fs. I know some problems regarding snapshots are now resolved. The problems I'm seeing in 6.1 are: 1. After writing a lot if data/files to a filesystem with soft-updates enabled, then unmounting or remounting it: update error: /: blocks n files y It appears others have seen this too. It appears that an attempt to flush the buffers to disk as part of the unmount failed. Is this a known issue? Could you describe how to reproduce the error? That would be really useful. HTH, Nikos ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
fsck, soft-updates, and snapshots
Hi, I almost sent this to freebsd-hackers, so please direct me there is that list is more appropriate. I am seeing some problems with soft-updates and fsck (I believe) so I have a couple of questions. Can anyone tell me src/sys/ufs/ffs/README.softupdates and README.snapshots are up to date? The former is dated June 2000, so it's almost 7 years old. The snapshots readme is only 2 years younger and mentions the code being alpha-test. The problems I'm seeing in 6.1 are: 1. After writing a lot if data/files to a filesystem with soft-updates enabled, then unmounting or remounting it: update error: /: blocks n files y It appears others have seen this too. It appears that an attempt to flush the buffers to disk as part of the unmount failed. Is this a known issue? 2. I was planning to use soft-updates and then have fsck run in the background if necessary on reboot to reclaim any lost blocks (as suggested by Kirk's whitepaper(s)). Normally the filesystem in question would be mounted read-only, but will be mounted read-write at times and remounted read-only. Will a background fsck work on a filesystem mounted read-only? Will it operate on a snapshot and if so, can or how does it replace the existing filesystem when the snapshot is corrected? Any advice much appreciated. Thanks, Dave -- Software Engineer Secure and Mobile Connectivity Nokia Enterprise Solutions +61 7 5553 8307 (office) +61 412 728 222 (cell) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: clean old portsnap snapshots?
Joe Auty wrote: What is the best mechanism for deleting old portsnap shots to free up some space? Or, is this supposed to be handled automatically? It should be handled automatically. Colin Percival ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
clean old portsnap snapshots?
Hello, What is the best mechanism for deleting old portsnap shots to free up some space? Or, is this supposed to be handled automatically? --- Joe Auty NetMusician: web publishing software for musicians http://www.netmusician.org [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Snapshots - can we use is for cloning disk(full restore to a newhard disk)
On 2/28/06, Gayn Winters [EMAIL PROTECTED] wrote: [mailto:[EMAIL PROTECTED] On Behalf Of Iantcho Vassilev Sent: Tuesday, February 28, 2006 2:46 AM To: FreeBSD Questions Subject: Snapshots - can we use is for cloning disk(full restore to a newhard disk) Guys, i searched the web and the mailing lists about this topic,but i really didn`t find any interesting thing.. Can i use the snapshots for full restore and if yes how can we do that? The part that everyone is referring to the snapshots is the fcsk you can run on it while the filesystem is working also.. You ought to be able to clone a partition from a snapshot via backup (from the snapshot) | restore (to another disk). Doing this for every partition should clone the FreeBSD system as of the snapshot. I've never tried it, however, and I'd be interested in hearing from someone who has done it successfully. ... Yea.. That`s the idea... Maybe dd copy from the snapshot and then growfs if the hard disk is bigger... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Snapshots - can we use is for cloning disk(full restore to a new hard disk)
Hello to everyone! Guys, i searched the web and the mailing lists about this topic,but i really didn`t find any interesting thing.. Can i use the snapshots for full restore and if yes how can we do that? The part that everyone is referring to the snapshots is the fcsk you can run on it while the filesystem is working also.. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Snapshots - can we use is for cloning disk(full restore to a newhard disk)
[mailto:[EMAIL PROTECTED] On Behalf Of Iantcho Vassilev Sent: Tuesday, February 28, 2006 2:46 AM To: FreeBSD Questions Subject: Snapshots - can we use is for cloning disk(full restore to a newhard disk) Guys, i searched the web and the mailing lists about this topic,but i really didn`t find any interesting thing.. Can i use the snapshots for full restore and if yes how can we do that? The part that everyone is referring to the snapshots is the fcsk you can run on it while the filesystem is working also.. You ought to be able to clone a partition from a snapshot via backup (from the snapshot) | restore (to another disk). Doing this for every partition should clone the FreeBSD system as of the snapshot. I've never tried it, however, and I'd be interested in hearing from someone who has done it successfully. -gayn Bristol Systems Inc. 714/532-6776 www.bristolsystems.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
snapshots on large filesystems
Hello, Considering a PC server running FreeBSD with 4 400 GB hard drives attached to a hardware raid controller doing raid-5. So this will present itself to the OS as a 1.2TB filesystem. Any comments on taking one or multiple snapshots of a filesystem of this size ? Given current disk capacities, I would not exactly consider this 1.2TB filesystem a large one ... any comments on say ... a 6-8 TB filesystem and making one or more snapshots of it ? Assume they are marginally busy - perhaps a 5-10% data turnover per day... Thanks. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: snapshots on large filesystems
Not enough info to be entirely informative in my reply, but I'd look to rsync or something similar... copy data/snapshot to another machine. We're running a similar setup here using 250GB S-ATA RAID Edition drives and 3Ware Escalade 9000-series controllers, a second machine simply rsync's the data from the first nightly... was cheaper to have whole second raid setup and dedicated gigabit ethernet from one machine to the other than was to invest in tape backup devices/media. In addition we do a bi-monthly snapshot on an external USB drive (via tar direct to device... not reccomend cause' it's slow, looking for a faster way myself to do that). With the cost of 200+GB drives, and applicable decent performing raid cards... it's just cheaper in most cases to mirror the data on another machine. - but that's just my two cents ;) -- Nathan Vidican [EMAIL PROTECTED] Windsor Match Plate Tool Ltd. http://www.wmptl.com/ user wrote: Hello, Considering a PC server running FreeBSD with 4 400 GB hard drives attached to a hardware raid controller doing raid-5. So this will present itself to the OS as a 1.2TB filesystem. Any comments on taking one or multiple snapshots of a filesystem of this size ? Given current disk capacities, I would not exactly consider this 1.2TB filesystem a large one ... any comments on say ... a 6-8 TB filesystem and making one or more snapshots of it ? Assume they are marginally busy - perhaps a 5-10% data turnover per day... Thanks. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
two quick conceptual questions RE: rsync (and rsyncing snapshots)
Hello, I have started using rsync somewhat extensively and had two questions regarding its operation. First, how does rsync respond to, and perform, when the source filesystem is under very heavy change ? If I have a filesystem that I want to rsync up to a backup server, but that filesystem is _very busy_ with the creation, destruction and changing of files, how well does rsync perform, and how much does it interfere with the performance of the underlying filesystem that it is sending up to the backup server ? Related: it occurs to me that perhaps it would be better to snapshot the filesystem, mount the snapshot, and then rsync the snapshot. On the other hand, the filesystem is continuously altering the snapshot as files are destroyed or changed ... so perhaps this does not gain anything. Is rsyncing a snapshot of a busy filesystem always, ever or never easier than rsyncing the busy filesystem itself ? Finally, am I correct that there are _only two_ rsync comparison methods - the default checksum method, and the --size-only method ? Am I correct that rsync _always_ looks at the timestamp first, and then applies either checksum or size comparison ONLY IF the timestamps are different ? Thanks a lot. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: two quick conceptual questions RE: rsync (and rsyncing snapshots)
user wrote: First, how does rsync respond to, and perform, when the source filesystem is under very heavy change ? If I have a filesystem that I want to rsync up to a backup server, but that filesystem is _very busy_ with the creation, destruction and changing of files, how well does rsync perform, and how much does it interfere with the performance of the underlying filesystem that it is sending up to the backup server ? rsync complains when the filesystem changes underneath it, but it will continue to run. On the other hand, rsync is not going to safely maintain the referential integrity of a complex file like a live database, but it's okay for most other things including mbox's. Rsync imposes a significant workload if you are syncronizing a large tree of stuff which changes a lot, but it's efficient considering the size of the task. Related: it occurs to me that perhaps it would be better to snapshot the filesystem, mount the snapshot, and then rsync the snapshot. On the other hand, the filesystem is continuously altering the snapshot as files are destroyed or changed ... so perhaps this does not gain anything. Is rsyncing a snapshot of a busy filesystem always, ever or never easier than rsyncing the busy filesystem itself ? rsync'ing a snapshot is a fine idea. Finally, am I correct that there are _only two_ rsync comparison methods - the default checksum method, and the --size-only method ? Am I correct that rsync _always_ looks at the timestamp first, and then applies either checksum or size comparison ONLY IF the timestamps are different ? No, rsync checks both timestamp and size or checksum. -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: two quick conceptual questions RE: rsync (and rsyncing snapshots)
Chuck - thank you... On Tue, 1 Nov 2005, Chuck Swiger wrote: rsync complains when the filesystem changes underneath it, but it will continue to run. On the other hand, rsync is not going to safely maintain the referential integrity of a complex file like a live database, but it's okay for most other things including mbox's. Does it simply complain, or does that somehow alter (lessen) the integrity of the sync that is going on ? Related: it occurs to me that perhaps it would be better to snapshot the filesystem, mount the snapshot, and then rsync the snapshot. On the other hand, the filesystem is continuously altering the snapshot as files are destroyed or changed ... so perhaps this does not gain anything. Is rsyncing a snapshot of a busy filesystem always, ever or never easier than rsyncing the busy filesystem itself ? rsync'ing a snapshot is a fine idea. Ok - because _nothing_ would change, and thus rsync would not complain, etc. My gut is that while rsync performance might be increased, filesystem performance would be about the same, since all destructions and changes on the underlying filesystem are still being committed to the snapshot ... can you think of a reason why this would not only increase the rsync performance, but the overall FS performance while rsyncing ? Finally, am I correct that there are _only two_ rsync comparison methods - the default checksum method, and the --size-only method ? Am I correct that rsync _always_ looks at the timestamp first, and then applies either checksum or size comparison ONLY IF the timestamps are different ? No, rsync checks both timestamp and size or checksum. So you are saying even if the timestamps are identical, rsync will _still_ do either a size or checksum comparison ? That seems ... inefficient ? Is there a way to tell it if the timestamps are identical, just move on ? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: two quick conceptual questions RE: rsync (and rsyncing snapshots)
user wrote: Chuck - thank you... Sure. On Tue, 1 Nov 2005, Chuck Swiger wrote: rsync complains when the filesystem changes underneath it, but it will continue to run. On the other hand, rsync is not going to safely maintain the referential integrity of a complex file like a live database, but it's okay for most other things including mbox's. Does it simply complain, or does that somehow alter (lessen) the integrity of the sync that is going on ? The files which it notices are either copied or noticed as being missing. Any new files which get created after rsync does it's first scan are not going to be picked up later on by rsync. rsync'ing a snapshot is a fine idea. Ok - because _nothing_ would change, and thus rsync would not complain, etc. My gut is that while rsync performance might be increased, filesystem performance would be about the same, since all destructions and changes on the underlying filesystem are still being committed to the snapshot ... can you think of a reason why this would not only increase the rsync performance, but the overall FS performance while rsyncing ? No. The point of using snapshots is to address the integrity concern above, they don't do anything in particular to change the performance. If disk I/O is a significant concern to you, add more spindles, use RAID-1 or RAID-10 configurations, or some combination of the two. Finally, am I correct that there are _only two_ rsync comparison methods - the default checksum method, and the --size-only method ? Am I correct that rsync _always_ looks at the timestamp first, and then applies either checksum or size comparison ONLY IF the timestamps are different ? No, rsync checks both timestamp and size or checksum. So you are saying even if the timestamps are identical, rsync will _still_ do either a size or checksum comparison ? That seems ... inefficient ? Is there a way to tell it if the timestamps are identical, just move on ? What happens if a program appends some more data during the same second? rsync has to fstat() the file anyway which potentially involves a disk operation, once it's done so, comparing both timestamp and size doesn't take a significant amount longer to do. -c, --checksum skip based on checksum, not mod-time size -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ... - resolved, but two more Qs
user [EMAIL PROTECTED] writes: Folks, On Thu, 20 Oct 2005, Gayn Winters wrote: Imagine that each data block is marked with labels on change. It doesn't matter how many labels there are, there will be only one data block saved. In trying to follow this thread, I started looking around for a precise definition of snapshot. Man mksnap_ffs wasn't too helpful, and googling for snapshot etc. wasn't fruitful. I'm guessing that the original author of the thread (user at dhp.com) may also need such a definition. Can someone provide a pointer to a specification or at least an RFC-like paper? I found one: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/ufs/ffs/README.snapshot?rev=1.4 and further, I did some tests and discovered that what I was being told (by you folks) was indeed correct. No matter how many snapshots you have, the changes in blocks since the tiem before the first snapshot is only recorded in one of them. That is to say, if I do the following: - create 4 1gig /dev/zero filled files - create a snapshot - overwrite one of those 1gig files with /dev/random My free space will have decreased by 1gig. So far so good. If I then: - create a second snapshot - overwrite a different 1gig file with /dev/random My free space merely decreases by another 1gig. It makes sense to me now because it has occurred to me that since the second file had not changed between the creation of the first and second snapshot, there is no reason for _both_ snapshots to _both_ say this 1gig random file used to be filled with zeros - it would be redundant. So that's great ... but I am curious, how do they know ? I think my previous assumption (that the first _and_ the second snapshot file would _both_ have to record the change of file #2 from zero to random) was based on the notion that these snapshot files were totally autonomous and independent, and had no general organization behind them. If that was the case, then I am still fairly certain both snapshots would need to record the change of the second file. Yes, they both need to notice, but they can share the actual copy of the data. So what is the behind the scenes organization that makes it possible for the snapshot files to not duplicate data like that ? Without trying to give a whole course in filesystems (there are books available if you want to go in depth), the data in the file is held in a number of data blocks, but there is meta-data that tells where the data is. When a file is overwritten, the snapshots continue to use the old version of the meta-data, which continues to point to the old data, while the real filesystem creates a new meta-data container pointing to new data blocks. If you then make another snapshot, the snapshot will use the new meta-data and its associated underlying data. It's an application of the copy-on-write principle. http://en.wikipedia.org/wiki/Copy-on-write ALSO, I have noticed that if you: - dd 1gig /dev/zero file - create snapshot - overwrite that 1gig file with /dev/random (free space decreases by 1gig, as expected) - rewrite that 1gig file with /dev/zero again You _don't_ get that 1gig of free space back ... which surprises me, since it was all zeros before, and its all zeros now ... how does the snapshot know those are different zeros ? And what ramifications does this have for restoring, etc., if identical files do not get counted as identical in the snapshot ? The snapshot doesn't know what the bits in the file are. All it knows is that the file's data used to be, say in block 1857 and now the file's data are in block 1956. The fact that both blocks are identical is not detected. If you're really interested in this, I suggest reading a decent operating systems book. It's a lot easier to understand the specific implementation when you have a good grip on the standard terminology and principles. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ... - resolved, but two more Qs
On 21 Oct 2005, Lowell Gilbert wrote: The snapshot doesn't know what the bits in the file are. All it knows is that the file's data used to be, say in block 1857 and now the file's data are in block 1956. The fact that both blocks are identical is not detected. If you're really interested in this, I suggest reading a decent operating systems book. It's a lot easier to understand the specific implementation when you have a good grip on the standard terminology and principles. Thanks very much for your help. I am going to read a book or two - my plan was to start with the design adn implementation of the 4.4BSD OS, but I wanted to update it with more modern information - like snapshots, etc., which I will do with those URLs we have already posted RE: the snapshot work. If you have any others, let me know. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ... - resolved, but two more Qs
user [EMAIL PROTECTED] writes: On 21 Oct 2005, Lowell Gilbert wrote: The snapshot doesn't know what the bits in the file are. All it knows is that the file's data used to be, say in block 1857 and now the file's data are in block 1956. The fact that both blocks are identical is not detected. If you're really interested in this, I suggest reading a decent operating systems book. It's a lot easier to understand the specific implementation when you have a good grip on the standard terminology and principles. Thanks very much for your help. I am going to read a book or two - my plan was to start with the design adn implementation of the 4.4BSD OS, but I wanted to update it with more modern information - like snapshots, etc., which I will do with those URLs we have already posted RE: the snapshot work. If you have any others, let me know. Yes. Start with something more basic, because McKusick's books assume that you are already acquainted with the standard terminology. Tanenbaum's are the usual recommendations. And when you do get to McKusick, you'll do a lot better with the new Design and Implementation of the FreeBSD Operating System, which covers a lot of these recent improvements. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
FreeBSD UFS2 snapshots, and math ...
I am trying to budget some disk space for filesystems with snapshots enabled on them. The following is simplified - I am just trying to get my concepts in order: Let's say I have a filesystem, and on that filesystem I create a snapshot every single night, and every night I delete the snapshot from 5 nights ago. This means that at all times, I have four snapshots running on that filesystem, one from 1 day ago, one from 2 days ago, one from 3 days ago, and one from 4 days ago. Let's also assume that the percent change of the filesystem is 5% (every day 5% of the blocks in the filesystem are either changed or deleted). Does this mean that if that 5% change is a different 5% every day, that the one day ago snapshot will be size 5%_of_filesystem, and that the 2 day ago snapshot will be size 10%_of_filesystem, day 3 15% and day 4 20%, for a total of 50% of the total filesystem taken up with snapshot data ? Does that sound correct ? When I say that the 5% change is a different 5% every day, what I mean is that it is not the same files/data being altered every day, but rather there is 5% of new data changed every day, relative to the previous nights snapshot. The second question is this: If the 5% data changed per day is the _same_ 5% every day (perhaps changing the same table in a DB every day, or perhaps changing the same block of lines in a text file every day) does that mean that every day simply represents 5%_of_filesystem, for a total of 20% of the total filesystem in use at all times for snapshot data ? - Finally, are there any snapshot diag tools at all ? Like, something that reports snapshot sizes, percent of disk used for snapshots, and maybe even a way for me to actually calculate what the percent change for time period X is for a particular filsystem ? Thank you. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ...
On Thu, Oct 20, 2005 at 11:01:42AM -0400, user wrote: Finally, are there any snapshot diag tools at all ? Like, something that reports snapshot sizes, percent of disk used for snapshots, and maybe even a way for me to actually calculate what the percent change for time period X is for a particular filsystem ? I find sysutils/freebsd-snapshot quite useful, although it doesn't do everything you're asking for. Be advised there are issues with snapshots of large filesystems. http://www.freebsd.org/cgi/query-pr-summary.cgi?category=severity=priority=class=state=sort=nonetext=snapshotresponsible=multitext=originator=release= My experience shows that multiple snapshots of large filesystems, 30GB, causes the system to become unresponsive when a snap is created. If I limit the snapshot to 1/per large FS, then often the machine hangs on attempted reboots. -- Regards, Doug ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ...
Doug, On Thu, 20 Oct 2005, Doug Poland wrote: On Thu, Oct 20, 2005 at 11:01:42AM -0400, user wrote: Finally, are there any snapshot diag tools at all ? Like, something that reports snapshot sizes, percent of disk used for snapshots, and maybe even a way for me to actually calculate what the percent change for time period X is for a particular filsystem ? I find sysutils/freebsd-snapshot quite useful, although it doesn't do everything you're asking for. Thanks - I will check that out. Any comments on my math ? (changing the same 5% of the FS all the time vs. changing different 5%'s, and what that means for successive snapshots) ? thanks. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ...
user [EMAIL PROTECTED] writes: I am trying to budget some disk space for filesystems with snapshots enabled on them. The following is simplified - I am just trying to get my concepts in order: Let's say I have a filesystem, and on that filesystem I create a snapshot every single night, and every night I delete the snapshot from 5 nights ago. This means that at all times, I have four snapshots running on that filesystem, one from 1 day ago, one from 2 days ago, one from 3 days ago, and one from 4 days ago. Let's also assume that the percent change of the filesystem is 5% (every day 5% of the blocks in the filesystem are either changed or deleted). Does this mean that if that 5% change is a different 5% every day, that the one day ago snapshot will be size 5%_of_filesystem, and that the 2 day ago snapshot will be size 10%_of_filesystem, day 3 15% and day 4 20%, for a total of 50% of the total filesystem taken up with snapshot data ? No. One copy of each version of the file that exists in any snapshot. Regardless of how many snapshots it's in. Does that sound correct ? When I say that the 5% change is a different 5% every day, what I mean is that it is not the same files/data being altered every day, but rather there is 5% of new data changed every day, relative to the previous nights snapshot. The second question is this: If the 5% data changed per day is the _same_ 5% every day (perhaps changing the same table in a DB every day, or perhaps changing the same block of lines in a text file every day) does that mean that every day simply represents 5%_of_filesystem, for a total of 20% of the total filesystem in use at all times for snapshot data ? Whether it's the same data or not doesn't affect how much space you use. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ...
Hello, On 20 Oct 2005, Lowell Gilbert wrote: user [EMAIL PROTECTED] writes: Let's say I have a filesystem, and on that filesystem I create a snapshot every single night, and every night I delete the snapshot from 5 nights ago. This means that at all times, I have four snapshots running on that filesystem, one from 1 day ago, one from 2 days ago, one from 3 days ago, and one from 4 days ago. Let's also assume that the percent change of the filesystem is 5% (every day 5% of the blocks in the filesystem are either changed or deleted). Does this mean that if that 5% change is a different 5% every day, that the one day ago snapshot will be size 5%_of_filesystem, and that the 2 day ago snapshot will be size 10%_of_filesystem, day 3 15% and day 4 20%, for a total of 50% of the total filesystem taken up with snapshot data ? No. One copy of each version of the file that exists in any snapshot. Regardless of how many snapshots it's in. That doesn't make much sense to me ... if the snapshot keeps track of changed_data_since_snapshot_was_taken, then ... Well, think of it this way - let's say I have a 1G filesystem, which is filled with a single 500M text file. Now let's say I snapshot that FS. At this point, the snapshot takes up 0 bytes. Now let's say the next day I alter 10% (50M) of that single 500M file - now the snapshot takes up that exact same amount of space, namely, 50M. Now I create a second snapshot, which immediately yakes up 0 bytes. The next day, I change a totally different 50M of my text file ... so now, the first snapshot needs to keep track of yesterdays 50M of changes/deletions as well as todays, because todays operates on totally different disk blocks. So now 2-day-ago snapshot is size 100M, and the snapshot from one day ago is now 50M. I think my interpretation is correct ... can you look over my and your conclusions again ? The second question is this: If the 5% data changed per day is the _same_ 5% every day (perhaps changing the same table in a DB every day, or perhaps changing the same block of lines in a text file every day) does that mean that every day simply represents 5%_of_filesystem, for a total of 20% of the total filesystem in use at all times for snapshot data ? Whether it's the same data or not doesn't affect how much space you use. Yeah ... see, I think it does matter, for the reasons above ... if, as in this second example, I am changing the same blocks on disk every day, the snapshot just needs to keep track of them once, namely this is what they were during the snapshot, and you can change those same blocks all you want, I just need to keep track of what they were when you took the snapshot.. comments ? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ...
On 10/20/05, user [EMAIL PROTECTED] wrote: Hello, On 20 Oct 2005, Lowell Gilbert wrote: user [EMAIL PROTECTED] writes: Let's say I have a filesystem, and on that filesystem I create a snapshot every single night, and every night I delete the snapshot from 5 nights ago. This means that at all times, I have four snapshots running on that filesystem, one from 1 day ago, one from 2 days ago, one from 3 days ago, and one from 4 days ago. Let's also assume that the percent change of the filesystem is 5% (every day 5% of the blocks in the filesystem are either changed or deleted). Does this mean that if that 5% change is a different 5% every day, that the one day ago snapshot will be size 5%_of_filesystem, and that the 2 day ago snapshot will be size 10%_of_filesystem, day 3 15% and day 4 20%, for a total of 50% of the total filesystem taken up with snapshot data ? No. One copy of each version of the file that exists in any snapshot. Regardless of how many snapshots it's in. That doesn't make much sense to me ... if the snapshot keeps track of changed_data_since_snapshot_was_taken, then ... Well, think of it this way - let's say I have a 1G filesystem, which is filled with a single 500M text file. Now let's say I snapshot that FS. At this point, the snapshot takes up 0 bytes. Now let's say the next day I alter 10% (50M) of that single 500M file - now the snapshot takes up that exact same amount of space, namely, 50M. Now I create a second snapshot, which immediately yakes up 0 bytes. The next day, I change a totally different 50M of my text file ... so now, the first snapshot needs to keep track of yesterdays 50M of changes/deletions as well as todays, because todays operates on totally different disk blocks. So now 2-day-ago snapshot is size 100M, and the snapshot from one day ago is now 50M. I think my interpretation is correct ... can you look over my and your conclusions again ? The second question is this: If the 5% data changed per day is the _same_ 5% every day (perhaps changing the same table in a DB every day, or perhaps changing the same block of lines in a text file every day) does that mean that every day simply represents 5%_of_filesystem, for a total of 20% of the total filesystem in use at all times for snapshot data ? Whether it's the same data or not doesn't affect how much space you use. Yeah ... see, I think it does matter, for the reasons above ... if, as in this second example, I am changing the same blocks on disk every day, the snapshot just needs to keep track of them once, namely this is what they were during the snapshot, and you can change those same blocks all you want, I just need to keep track of what they were when you took the snapshot.. comments ? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] What makes you so reassured, I wonder. Imagine that each data block is marked with labels on change. It doesn't matter how many labels there are, there will be only one data block saved. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: FreeBSD UFS2 snapshots, and math ...
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew P. Sent: Thursday, October 20, 2005 12:35 PM To: user Cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD UFS2 snapshots, and math ... Imagine that each data block is marked with labels on change. It doesn't matter how many labels there are, there will be only one data block saved. In trying to follow this thread, I started looking around for a precise definition of snapshot. Man mksnap_ffs wasn't too helpful, and googling for snapshot etc. wasn't fruitful. I'm guessing that the original author of the thread (user at dhp.com) may also need such a definition. Can someone provide a pointer to a specification or at least an RFC-like paper? Thanks, -gayn ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: FreeBSD UFS2 snapshots, and math ... - resolved, but two more Qs
Folks, On Thu, 20 Oct 2005, Gayn Winters wrote: Imagine that each data block is marked with labels on change. It doesn't matter how many labels there are, there will be only one data block saved. In trying to follow this thread, I started looking around for a precise definition of snapshot. Man mksnap_ffs wasn't too helpful, and googling for snapshot etc. wasn't fruitful. I'm guessing that the original author of the thread (user at dhp.com) may also need such a definition. Can someone provide a pointer to a specification or at least an RFC-like paper? I found one: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/ufs/ffs/README.snapshot?rev=1.4 and further, I did some tests and discovered that what I was being told (by you folks) was indeed correct. No matter how many snapshots you have, the changes in blocks since the tiem before the first snapshot is only recorded in one of them. That is to say, if I do the following: - create 4 1gig /dev/zero filled files - create a snapshot - overwrite one of those 1gig files with /dev/random My free space will have decreased by 1gig. So far so good. If I then: - create a second snapshot - overwrite a different 1gig file with /dev/random My free space merely decreases by another 1gig. It makes sense to me now because it has occurred to me that since the second file had not changed between the creation of the first and second snapshot, there is no reason for _both_ snapshots to _both_ say this 1gig random file used to be filled with zeros - it would be redundant. So that's great ... but I am curious, how do they know ? I think my previous assumption (that the first _and_ the second snapshot file would _both_ have to record the change of file #2 from zero to random) was based on the notion that these snapshot files were totally autonomous and independent, and had no general organization behind them. If that was the case, then I am still fairly certain both snapshots would need to record the change of the second file. So what is the behind the scenes organization that makes it possible for the snapshot files to not duplicate data like that ? ALSO, I have noticed that if you: - dd 1gig /dev/zero file - create snapshot - overwrite that 1gig file with /dev/random (free space decreases by 1gig, as expected) - rewrite that 1gig file with /dev/zero again You _don't_ get that 1gig of free space back ... which surprises me, since it was all zeros before, and its all zeros now ... how does the snapshot know those are different zeros ? And what ramifications does this have for restoring, etc., if identical files do not get counted as identical in the snapshot ? thanks. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: FreeBSD UFS2 snapshots, and math ... - resolved, but two more Qs
-Original Message- From: user [mailto:[EMAIL PROTECTED] Sent: Thursday, October 20, 2005 1:51 PM To: Gayn Winters Cc: 'Andrew P.'; freebsd-questions@freebsd.org Subject: RE: FreeBSD UFS2 snapshots, and math ... - resolved, but two more Qs Folks, On Thu, 20 Oct 2005, Gayn Winters wrote: Imagine that each data block is marked with labels on change. It doesn't matter how many labels there are, there will be only one data block saved. In trying to follow this thread, I started looking around for a precise definition of snapshot. Man mksnap_ffs wasn't too helpful, and googling for snapshot etc. wasn't fruitful. I'm guessing that the original author of the thread (user at dhp.com) may also need such a definition. Can someone provide a pointer to a specification or at least an RFC-like paper? I found one: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/ufs/ffs/README.s napshot?rev=1.4 and further, I did some tests and discovered that what I was being told (by you folks) was indeed correct. No matter how many snapshots you have, the changes in blocks since the tiem before the first snapshot is only recorded in one of them. That is to say, if I do the following: - create 4 1gig /dev/zero filled files - create a snapshot - overwrite one of those 1gig files with /dev/random My free space will have decreased by 1gig. So far so good. If I then: - create a second snapshot - overwrite a different 1gig file with /dev/random My free space merely decreases by another 1gig. It makes sense to me now because it has occurred to me that since the second file had not changed between the creation of the first and second snapshot, there is no reason for _both_ snapshots to _both_ say this 1gig random file used to be filled with zeros - it would be redundant. So that's great ... but I am curious, how do they know ? I think my previous assumption (that the first _and_ the second snapshot file would _both_ have to record the change of file #2 from zero to random) was based on the notion that these snapshot files were totally autonomous and independent, and had no general organization behind them. If that was the case, then I am still fairly certain both snapshots would need to record the change of the second file. So what is the behind the scenes organization that makes it possible for the snapshot files to not duplicate data like that ? ALSO, I have noticed that if you: - dd 1gig /dev/zero file - create snapshot - overwrite that 1gig file with /dev/random (free space decreases by 1gig, as expected) - rewrite that 1gig file with /dev/zero again You _don't_ get that 1gig of free space back ... which surprises me, since it was all zeros before, and its all zeros now ... how does the snapshot know those are different zeros ? And what ramifications does this have for restoring, etc., if identical files do not get counted as identical in the snapshot ? thanks. I just finished skimming an old paper by McKusick on Soft Updates: http://www.usenix.org/publications/library/proceedings/usenix99/full_pap ers/mckusick/mckusick.pdf This paper is dated 1999. Does anyone know if it accurately reflects how soft updates and snapshots in FreeBSD 5.4 are implemented? If so, it would answer the above questions. -gayn ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD UFS2 snapshots, and math ...
On 10/21/05, Gayn Winters [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew P. Sent: Thursday, October 20, 2005 12:35 PM To: user Cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD UFS2 snapshots, and math ... Imagine that each data block is marked with labels on change. It doesn't matter how many labels there are, there will be only one data block saved. In trying to follow this thread, I started looking around for a precise definition of snapshot. Man mksnap_ffs wasn't too helpful, and googling for snapshot etc. wasn't fruitful. I'm guessing that the original author of the thread (user at dhp.com) may also need such a definition. Can someone provide a pointer to a specification or at least an RFC-like paper? Thanks, -gayn Here ya go: http://www.mckusick.com/softdep/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: GCC snapshots and the ports collection
Kris Kennaway wrote: On Sat, Sep 10, 2005 at 01:41:15PM +0200, K?vesd?n G?bor wrote: Hello, I have two issues with selecting the appropriate version of gcc: 1, There is the port net/verlihub, that needs gcc 3.3 that is broken under amd64. What solution do You recommend? You could try it with the system compiler, but chances are it depends on gcc 3.3 because later versions cannot compile it (they are stricter about the conforming code they will compile, particularly C++) I've tried it, but unfortunately it fails. If I download the official tarball and try to compile that, it succeeds, however. Accordingly, I think the port can be fixed to compile with the stock compiler, but unfortunately I can't figure out what the problem is. 2, There are gcc snapshots in the ports collection like lang/gcc34, lang/gcc40, lang/gcc41, but there aren't releaes, just snapshots. Are these gcc snapshots as reliable as the releases are? Can I use for instance lang/gcc34 for production goals instead of the stock compiler, or is it just for development/testing usage? Not for building world, but you can use them for your own purposes. But if these aren't so reliable as the gcc releases are, why don't we have the releases in the ports collection instead of the snapshots? Or has anybody thought of porting the official releases as lang/gcc401 or something like that? Gabor Kovesdan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: GCC snapshots and the ports collection
On Sun, Sep 11, 2005 at 03:27:11PM +0200, K?vesd?n G?bor wrote: Kris Kennaway wrote: On Sat, Sep 10, 2005 at 01:41:15PM +0200, K?vesd?n G?bor wrote: Hello, I have two issues with selecting the appropriate version of gcc: 1, There is the port net/verlihub, that needs gcc 3.3 that is broken under amd64. What solution do You recommend? You could try it with the system compiler, but chances are it depends on gcc 3.3 because later versions cannot compile it (they are stricter about the conforming code they will compile, particularly C++) I've tried it, but unfortunately it fails. If I download the official tarball and try to compile that, it succeeds, however. Accordingly, I think the port can be fixed to compile with the stock compiler, but unfortunately I can't figure out what the problem is. Talk to the port maintainer or developers. 2, There are gcc snapshots in the ports collection like lang/gcc34, lang/gcc40, lang/gcc41, but there aren't releaes, just snapshots. Are these gcc snapshots as reliable as the releases are? Can I use for instance lang/gcc34 for production goals instead of the stock compiler, or is it just for development/testing usage? Not for building world, but you can use them for your own purposes. But if these aren't so reliable as the gcc releases are, why don't we have the releases in the ports collection instead of the snapshots? Or has anybody thought of porting the official releases as lang/gcc401 or something like that? Ditto. Kris pgpbtcqZry5fw.pgp Description: PGP signature
GCC snapshots and the ports collection
Hello, I have two issues with selecting the appropriate version of gcc: 1, There is the port net/verlihub, that needs gcc 3.3 that is broken under amd64. What solution do You recommend? I haven't used the compatibility layer yet, but what if I make buildworld/installworld to enable the compatibility layer? Is there any way to cross-compile ports similar to make TARGET_ARCH=i386 buildworld? Is this macro usable with ports collection like cd /usr/ports/lang/gcc33 make TARGET_ARCH=i386 install? 2, There are gcc snapshots in the ports collection like lang/gcc34, lang/gcc40, lang/gcc41, but there aren't releaes, just snapshots. Are these gcc snapshots as reliable as the releases are? Can I use for instance lang/gcc34 for production goals instead of the stock compiler, or is it just for development/testing usage? Cheers, Gabor Kovesdan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: GCC snapshots and the ports collection
Kövesdán Gábor wrote: I have two issues with selecting the appropriate version of gcc: 1, There is the port net/verlihub, that needs gcc 3.3 that is broken under amd64. What solution do You recommend? Fix net/verlihub to not depend on a specific version of gcc. 2, There are gcc snapshots in the ports collection like lang/gcc34, lang/gcc40, lang/gcc41, but there aren't releaes, just snapshots. Are these gcc snapshots as reliable as the releases are? No. Can I use for instance lang/gcc34 for production goals instead of the stock compiler, or is it just for development/testing usage? You can use a snapshot compiler to build anything you want, although it is recommended that you use the system compiler when building the system itself. (Unfortunately, the GCC folks have been breaking the C++ ABI from release to release pretty regularly, which means that sufficiently different gcc revisions cannot link each other's C++ object code or libraries.) -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: GCC snapshots and the ports collection
On Sat, Sep 10, 2005 at 01:41:15PM +0200, K?vesd?n G?bor wrote: Hello, I have two issues with selecting the appropriate version of gcc: 1, There is the port net/verlihub, that needs gcc 3.3 that is broken under amd64. What solution do You recommend? You could try it with the system compiler, but chances are it depends on gcc 3.3 because later versions cannot compile it (they are stricter about the conforming code they will compile, particularly C++) I haven't used the compatibility layer yet, but what if I make buildworld/installworld to enable the compatibility layer? Is there any way to cross-compile ports similar to make TARGET_ARCH=i386 buildworld? Is this macro usable with ports collection like cd /usr/ports/lang/gcc33 make TARGET_ARCH=i386 install? No. 2, There are gcc snapshots in the ports collection like lang/gcc34, lang/gcc40, lang/gcc41, but there aren't releaes, just snapshots. Are these gcc snapshots as reliable as the releases are? Can I use for instance lang/gcc34 for production goals instead of the stock compiler, or is it just for development/testing usage? Not for building world, but you can use them for your own purposes. Kris pgpcuDidDBe4G.pgp Description: PGP signature
Re: rsync and moving files [Re: backup w/ snapshots]
* Charles Swiger [2005-08-30 10:49 -0400] On Aug 30, 2005, at 3:32 AM, Svein Halvor Halvorsen wrote: Yes, that's correct! But let's say I keep more than one snapshot around. I maybe didn't mention this, but this the sole purpose of using snapshots; for me to have more full backups laying around. A snapshot on the same disk does not qualify as a reliable backup of your data. Using rsync to copy a tree of stuff to another machine would. Please read the entire thread. I use rsync to mirror my disks remotely, then make snapshots on that remote computer. The snapshots are mounted read-only and nfs-exported back to the original computer. This satisfies both the need for offsite sorage of backups, the need for invremental backups and the need for all previous backups to be randomly accessible from the original computer. Thanks for your consern, though. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
* Garance A Drosihn [2005-08-30 12:50 -0400] Fwiw, I understand the problem you're trying to describe. And the basic issue is that rsync keeps no information between separate runs of it. It has no way of knowing that a given file on the source volume used to be at a different location. It does not even know that the destination volume was sync'ed by a previous run of rsync, so it does not even know that the file at the old location on the destination is the same as the file at the old location on the source. It knows nothing more than the information it has at the moment of any given run of rsync. You could kinda fudge that information for rsync by creating a lot of hard links, but that is probably going to create more of a mess than it will solve. So, you're left with doing something else outside of rsync. The script you are suggesting would probably be fairly easy to write in something like ruby, perl, or python. Use a key made up of the inode number + lastchange date, or maybe inode number + file size. Then save away the key-to-filename(s) mapping for every file. On the next run of rsync, see which files have moved on the source directory. If the destination volume has a file at the old location which matches the file-size or lastchange date (depending on which key you used...), then move it to the new location on the destination volume. Thanks! I think I will try to implement this, then! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
* Greg Barniskis [2005-08-29 11:45 -0500] Eh? Bad assumptions about snapshots, I think. If a snapshot occupied even a tenth of the space of the data that it represented, we would quickly fill all our disks and the snapshot technology would be almost as painful as useful. A snapshot is essentially only an index of occupied disk space, not a copy of the actual data, and a snapshot is therefore much, much, much, much smaller than the data files that have changed. Read the relevant man pages and handbook sections again, and test your assumptions by measuring the actual change in snapshot size. I don't think your perceived problem really exists. Yes, that's correct! But let's say I keep more than one snapshot around. I maybe didn't mention this, but this the sole purpose of using snapshots; for me to have more full backups laying around. If I change the disk alot between snapshots. Eg. I rsync moved files (yes, within tha same fs), this will result in alot of file deletion and creation. Next, when I make the snapshot, a new list of occupied diskspace will be made, and all of these blocks will be marked in use, and therefore take up alot of diskspace. In reality the information change between the two snapshots, didn't change much at all, but the effect remains: my disk cannot longer store two snapshots (unless the backup disk is twice as large, which it is not). The solution: Somehow, I need to mirror all the move ops on the remote system before doing the rsync. This could probably be done by making a hash table of inodes/filenames pairs (or triplets, etc) each time i sync. Then the next time, I could compare the old table with the new, to find out which files are the same only with new names, then find those names on the remote system, change them to the new ones, and then rsyncing. If the inodes are recycled for brand new files between syncs, I don't think that would be a problem. The following rsync-job would recognize the diffs and sync that, which it would have done anyway, if the file is new. What do you think? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
* Norberto Meijome [2005-08-30 02:14 +1000] I take your word wrt to how it works. Assuming of course that you move within the same filesystem. Yes, I'm talking about the same filsystem. (touche). yup, that's what would happenbut tha's the nature of the beast :) don't keep too many snapshots ? ;) it'd be great if you could keep a log of all local-mv operations,and then replay them remotely via ssh. Yes, I thought about that myself. Only I thought I'd keep a list of filename/inode pairs from each sync, so before I do a sync I could compare the lists to find out which files appears to be the same, only with a new name. Then rename those files remotely. In cases where a inode-match does not represent a relink, but just plain inode recycling, so what? Rsync will make the new file up to date. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
* Bob Johnson [2005-08-29 12:44 -0400] Use a ggated(8) + ggatec(8) pair to establish a remote volume that looks local, then use gmirror to make it a mirror of the local drive. The big gotcha is that ggated/c only moves i/o requests and data via the net, it doesn't move ioctls, so some things just won't work remotely. Or at least, that's what I've read. Do you think this is allright for a 4M/640K link? The upstrem bandwith to the backupserver is 4 Mbps. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
Svein Halvor Halvorsen [EMAIL PROTECTED] writes: Only I thought I'd keep a list of filename/inode pairs from each sync, so before I do a sync I could compare the lists to find out which files appears to be the same, only with a new name. Doesn't dump(8)/restore(8) do pretty much this? But less crudely... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
Svein Halvor Halvorsen wrote: * Greg Barniskis [2005-08-29 11:45 -0500] Eh? Bad assumptions about snapshots, I think. If a snapshot occupied even a tenth of the space of the data that it represented, we would quickly fill all our disks and the snapshot technology would be almost as painful as useful. A snapshot is essentially only an index of occupied disk space, not a copy of the actual data, and a snapshot is therefore much, much, much, much smaller than the data files that have changed. Read the relevant man pages and handbook sections again, and test your assumptions by measuring the actual change in snapshot size. I don't think your perceived problem really exists. Yes, that's correct! But let's say I keep more than one snapshot around. I maybe didn't mention this, but this the sole purpose of using snapshots; for me to have more full backups laying around. Ah. That does change things a bit, I guess. A previous post indicated file renames and replication followed by taking a new snapshot, and I thought it was implied your older snapshots were going away. If I change the disk alot between snapshots. Eg. I rsync moved files (yes, within tha same fs), this will result in alot of file deletion and creation. Next, when I make the snapshot, a new list of occupied diskspace will be made, and all of these blocks will be marked in use, and therefore take up alot of diskspace. In reality the information change between the two snapshots, didn't change much at all, but the effect remains: my disk cannot longer store two snapshots (unless the backup disk is twice as large, which it is not). The solution: Somehow, I need to mirror all the move ops on the remote system before doing the rsync. This could probably be done by making a hash table of inodes/filenames pairs (or triplets, etc) each time i sync. Then the next time, I could compare the old table with the new, to find out which files are the same only with new names, then find those names on the remote system, change them to the new ones, and then rsyncing. If the inodes are recycled for brand new files between syncs, I don't think that would be a problem. The following rsync-job would recognize the diffs and sync that, which it would have done anyway, if the file is new. What do you think? This is admittedly beyond my ken, at least within the limited number of brain cycles I can offer to the problem. Hopefully someone else will provide clues for you. Personally, I think you're violating the KISS principle unless there's a really compelling need to keep your previous file system states accessible online. Dumping older states to offline media and reclaiming that space would be my first order of business, but that's just me. Or just buy some whopping big disks appropriate to the task, since that's generally cheaper than admin time to create workarounds (unless you just consider this fun =). Good luck, -- Greg Barniskis, Computer Systems Integrator South Central Library System (SCLS) Library Interchange Network (LINK) gregb at scls.lib.wi.us, (608) 266-6348 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
On Aug 30, 2005, at 3:32 AM, Svein Halvor Halvorsen wrote: Yes, that's correct! But let's say I keep more than one snapshot around. I maybe didn't mention this, but this the sole purpose of using snapshots; for me to have more full backups laying around. A snapshot on the same disk does not qualify as a reliable backup of your data. Using rsync to copy a tree of stuff to another machine would. -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
At 9:32 AM +0200 8/30/05, Svein Halvor Halvorsen wrote: The solution: Somehow, I need to mirror all the move ops on the remote system before doing the rsync. This could probably be done by making a hash table of inodes/filenames pairs (or triplets, etc) each time i sync. Then the next time, I could compare the old table with the new, to find out which files are the same only with new names, then find those names on the remote system, change them to the new ones, and then rsyncing. Fwiw, I understand the problem you're trying to describe. And the basic issue is that rsync keeps no information between separate runs of it. It has no way of knowing that a given file on the source volume used to be at a different location. It does not even know that the destination volume was sync'ed by a previous run of rsync, so it does not even know that the file at the old location on the destination is the same as the file at the old location on the source. It knows nothing more than the information it has at the moment of any given run of rsync. You could kinda fudge that information for rsync by creating a lot of hard links, but that is probably going to create more of a mess than it will solve. So, you're left with doing something else outside of rsync. The script you are suggesting would probably be fairly easy to write in something like ruby, perl, or python. Use a key made up of the inode number + lastchange date, or maybe inode number + file size. Then save away the key-to-filename(s) mapping for every file. On the next run of rsync, see which files have moved on the source directory. If the destination volume has a file at the old location which matches the file-size or lastchange date (depending on which key you used...), then move it to the new location on the destination volume. vague_rambling Hmm. Thinking about this a little more, it's probably possible for rsync to catch some of these cases itself. It would require some coding changes to rsync, but it could take the list of files that it is deleting, compare it to the list of files that it is adding, and if the MD5-checksum + size of some to-be-deleted file is the same as some to-be-added file, it could try doing a 'mv' of that file before it does the remainder of its processing. I wonder how hard that would be to do. /vague_rambling -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
Charles Swiger [EMAIL PROTECTED] writes: On Aug 30, 2005, at 3:32 AM, Svein Halvor Halvorsen wrote: Yes, that's correct! But let's say I keep more than one snapshot around. I maybe didn't mention this, but this the sole purpose of using snapshots; for me to have more full backups laying around. A snapshot on the same disk does not qualify as a reliable backup of your data. No, but it is convenient to restore from, when it's intact. This is actually a very common case; accidental file deletions. Using rsync to copy a tree of stuff to another machine would. And as long as one is doing that, there's no reason not to use snapshots as well. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Snapshots
Hi all...I'm having a problem using snapshots...which I discovered when I tried a system backup using dump. I've got a 283Gb partition, and the system was trying to create the snapshot for 12 hours. I'm on 5.4-RELEASE. Should this be taking this long? My gut tells me no...cause it'd be foolish if it did. Any ideas/suggestions? Thanks in advance, --Brian -- _-=-_-=-_-=-_-=-_-=-_-=-_-=-_-=-_-=-_-=-_-=-_ Brian McCann Systems Network Administrator, K12USA I don't have to take this abuse from you -- I've got hundreds of people waiting to abuse me. -- Bill Murray, Ghostbusters ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Snapshots
On 8/30/05, Brian McCann [EMAIL PROTECTED] wrote: Hi all...I'm having a problem using snapshots...which I discovered when I tried a system backup using dump. I've got a 283Gb partition, and the system was trying to create the snapshot for 12 hours. I'm on 5.4-RELEASE. Should this be taking this long? My gut tells me no...cause it'd be foolish if it did. Any ideas/suggestions? No, it shouldn't. It appears that there is some problem with the snapshot process on large filesystems in 5.4-RELEASE and perhaps others, but I don't know how aggressively it is being investigated, or whether a solution has already been found. There have been several posts about this in the past month or two, if you search the archives you might find better information about it. I remember there was conjecture about the possible cause (e.g. insufficient temporary storage space for the inode list), but I don't remember if this led to a workable solution. - Bob ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
rsync and moving files [Re: backup w/ snapshots]
* Svein Halvor Halvorsen [2005-08-28 23:53 +0200] Does this sound reasonable? Is there any precautions I should take? Are there any other tools better suited for the task at hand? I'm responding to my own message. Let's say I happen to move all music from /music/artist - album/ to /music/artist/album. Even though a local snapshot would handle this well, rsync would create new files on the remote machine, and when I then take a snapshot there, it will be HUGE! Can I resolve this? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
Svein Halvor Halvorsen wrote: * Svein Halvor Halvorsen [2005-08-28 23:53 +0200] Does this sound reasonable? Is there any precautions I should take? Are there any other tools better suited for the task at hand? I'm responding to my own message. Let's say I happen to move all music from /music/artist - album/ to /music/artist/album. Even though a local snapshot would handle this well, rsync would create new files on the remote machine, and when I then take a snapshot there, it will be HUGE! isn't that the whole point of having a backup? to have *another* copy of your files? and I guess that yes, if the files are new in the remote system, when you take a snapshot the difference with the previous snapshot will be the size of the new data (only guessing from how snapshots work in Linux, so feel free to flame ..err..correct me :) ) Beto ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
On 8/29/05, Norberto Meijome [EMAIL PROTECTED] wrote: Svein Halvor Halvorsen wrote: * Svein Halvor Halvorsen [2005-08-28 23:53 +0200] Does this sound reasonable? Is there any precautions I should take? Are there any other tools better suited for the task at hand? I'm responding to my own message. Let's say I happen to move all music from /music/artist - album/ to /music/artist/album. Even though a local snapshot would handle this well, rsync would create new files on the remote machine, and when I then take a snapshot there, it will be HUGE! isn't that the whole point of having a backup? to have *another* copy of your files? and I guess that yes, if the files are new in the remote system, when you take a snapshot the difference with the previous snapshot will be the size of the new data (only guessing from how snapshots work in Linux, so feel free to flame ..err..correct me :) ) Beto ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] cat /usr/ports/sysutils/rsnapshot/pkg-descr -Erik- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
* Norberto Meijome [2005-08-30 00:32 +1000] isn't that the whole point of having a backup? to have *another* copy of your files? Well, yes and no. The idea is that I have a main computer that I want to backup. I want the backup to be (a) remote, (b) incremental and (c) random-accessible. So I thought that every day my backup-server could rsync my main computer, creating a mirror of the relevant directory trees. Then, as soon as the rsync job completes, it takes a snapshot of the filesystem. This snapshot could be mounted r/o and nfs-exported back to the original computer. Now: If I have a file /foo/test on my main computer. After the first rsync-job this file will be copied, assigned an inode and put on the disk somewhere. If I change this file, a local snapshot will be smart enough to just store the changed sectors that this file now occupies. But: If I move the file from /foo/test to /bar/test on my main computer, rsync will create a BRAND NEW FILE in /bar (and delete the file in /foo, since I used the --delete option). Now this NEW file will have a new inode, and cover new sectors on disk. The snapshot will then tak considerable more diskspace. If I move a large directory tree this way, this will occupy huge amounts of diskspace. If I however, make the snapshot on my local disk, this is not a problem, as on this local filesystem /bar/test is not a new file. So how can I make rsync know that the files were just moved (renamed, relinked), and make rsync reflect this fact on the remote mirror? and I guess that yes, if the files are new in the remote system, when you take a snapshot the difference with the previous snapshot will be the size of the new data The files aren't new. Their names are! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
* Hornet [2005-08-29 11:11 -0400] cat /usr/ports/sysutils/rsnapshot/pkg-descr It seems this is just a wrapper around the tools I was already planning on using. In this regard, it's a nice port. But won't this perl-script suffer for tha same shortcommings that rsync will? Or does it use rsync in more clever ways that I do? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
Svein Halvor Halvorsen [EMAIL PROTECTED] writes: * Svein Halvor Halvorsen [2005-08-28 23:53 +0200] Does this sound reasonable? Is there any precautions I should take? Are there any other tools better suited for the task at hand? I'm responding to my own message. Let's say I happen to move all music from /music/artist - album/ to /music/artist/album. Even though a local snapshot would handle this well, rsync would create new files on the remote machine, and when I then take a snapshot there, it will be HUGE! Can I resolve this? dump/restore is the only method I can think of offhand that will handle this problem. And even then, those huge moves have to stay within a single filesystem. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
Svein Halvor Halvorsen wrote: * Norberto Meijome [2005-08-30 00:32 +1000] isn't that the whole point of having a backup? to have *another* copy of your files? Well, yes and no. The idea is that I have a main computer that I want to backup. I want the backup to be (a) remote, (b) incremental and (c) random-accessible. So I thought that every day my backup-server could rsync my main computer, creating a mirror of the relevant directory trees. Then, as soon as the rsync job completes, it takes a snapshot of the filesystem. This snapshot could be mounted r/o and nfs-exported back to the original computer. yes, that sounds like it would meet your criteria. Now: If I have a file /foo/test on my main computer. After the first rsync-job this file will be copied, assigned an inode and put on the disk somewhere. If I change this file, a local snapshot will be smart enough to just store the changed sectors that this file now occupies. I take your word wrt to how it works. Assuming of course that you move within the same filesystem. But: If I move the file from /foo/test to /bar/test on my main computer, ( /foo/ and /bar/ being in the same filesystem) rsync will create a BRAND NEW FILE in /bar (and delete the file in /foo, since I used the --delete option). Now this NEW file will have a new inode, and cover new sectors on disk. The snapshot will then tak considerable more diskspace. If I move a large directory tree this way, this will occupy huge amounts of diskspace. (touche). yup, that's what would happenbut tha's the nature of the beast :) don't keep too many snapshots ? ;) it'd be great if you could keep a log of all local-mv operations,and then replay them remotely via ssh. hack replace mv with your own version which does local-mv and either does remote-mv over ssh or sends a msg to a service to perform the transaction/hack (yes, there may be other conditions that trigger the same effect as an mv...I just dont know which ones) I guess the proper way to do this (if you are REALLY REALLY worried about that extra spaced used for snapshots in the remote site) would be to implement a GEOM class that knows about the remote site and the 'mv' condition and communicate to the remote end. So how can I make rsync know that the files were just moved (renamed, relinked), and make rsync reflect this fact on the remote mirror? rsync would then be the wrong tool for the job. I would suggest that you just get more or larger drives for the remote site and live with the waste :) (though the GEOM class would be cool :D ) hmm...what about network operating systems like AFS or CODA (not that I know much about them, I just read some stuff on those being distruted,etc..) Let us know how you solve this. Regards, Beto ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
On 8/29/05, Norberto Meijome [EMAIL PROTECTED] wrote: Svein Halvor Halvorsen wrote: * Norberto Meijome [2005-08-30 00:32 +1000] I guess the proper way to do this (if you are REALLY REALLY worried about that extra spaced used for snapshots in the remote site) would be to implement a GEOM class that knows about the remote site and the 'mv' condition and communicate to the remote end. In principle, it already exists. In practice, it might need more work. Use a ggated(8) + ggatec(8) pair to establish a remote volume that looks local, then use gmirror to make it a mirror of the local drive. The big gotcha is that ggated/c only moves i/o requests and data via the net, it doesn't move ioctls, so some things just won't work remotely. Or at least, that's what I've read. [...] hmm...what about network operating systems like AFS or CODA (not that I know much about them, I just read some stuff on those being distruted,etc..) Although CODA would probably work, I think the GEOM solution, if it works, would be far easier to implement. Let us know how you solve this. Regards, Beto ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rsync and moving files [Re: backup w/ snapshots]
Svein Halvor Halvorsen wrote: But: If I move the file from /foo/test to /bar/test on my main computer, rsync will create a BRAND NEW FILE in /bar (and delete the file in /foo, since I used the --delete option). Now this NEW file will have a new inode, and cover new sectors on disk. The snapshot will then tak considerable more diskspace. If I move a large directory tree this way, this will occupy huge amounts of diskspace. Eh? Bad assumptions about snapshots, I think. If a snapshot occupied even a tenth of the space of the data that it represented, we would quickly fill all our disks and the snapshot technology would be almost as painful as useful. A snapshot is essentially only an index of occupied disk space, not a copy of the actual data, and a snapshot is therefore much, much, much, much smaller than the data files that have changed. Read the relevant man pages and handbook sections again, and test your assumptions by measuring the actual change in snapshot size. I don't think your perceived problem really exists. -- Greg Barniskis, Computer Systems Integrator South Central Library System (SCLS) Library Interchange Network (LINK) gregb at scls.lib.wi.us, (608) 266-6348 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
backup w/ snapshots
I'm thinking about using snapshots as a kind of backup-mechanism, in order
to restore accidentally deleted files. Also, in order to avoid losing data
in case of a fire, etc., I'd like to store the backup off-site. I'm
thinking about using rsync to syncronize the relevant filesystems to the
off-site backup-server eg. every day, then taking a snapshot of the remote
filesystem, mount it as /export/backup/{date} and then nfs-exporting that
filesystem to the first computer again.
This way I'd have eg. the /home filsystem mirrored in /backup/{date}/home,
and my /etc in /backup/{date}/etc. I could symlink /backup/yesterday or
/backup/latest to the correct date.
The network link between the two computers are about 4 Mbps downstream,
and 640 Kbps upstream. That is; to take a backup is considerable faster
than to restore a file.
Does this sound reasonable? Is there any precautions I should take? Are
there any other tools better suited for the task at hand?
SVein Halvor
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Making UFS snapshots
On 2005-08-17 16:32, Daniel Feenberg [EMAIL PROTECTED] wrote: I notice on this list that Garance Drosehn http://docs.FreeBSD.org/cgi/mid.cgi?p06230924bf1c752ccf7f reports making a snapshot of a 4 gigabyte filesystem in less than one second. We have a 859 gigabyte filesystem and snapshots take about 75 minutes to complete. Making a snapshot is not very slow if the disk is relatively idle at the time. Perhaps this is what's biting you? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Making UFS snapshots
On Thu, 18 Aug 2005, Giorgos Keramidas wrote: On 2005-08-17 16:32, Daniel Feenberg [EMAIL PROTECTED] wrote: I notice on this list that Garance Drosehn http://docs.FreeBSD.org/cgi/mid.cgi?p06230924bf1c752ccf7f reports making a snapshot of a 4 gigabyte filesystem in less than one second. We have a 859 gigabyte filesystem and snapshots take about 75 minutes to complete. Making a snapshot is not very slow if the disk is relatively idle at the time. Perhaps this is what's biting you? The computer and the disk system is otherwise idle - no activity other than taking the snapshot. Since the original posting I found Dr McKusik's 1999 Usenix paper describing snapshots which suggests the time for taking a snapshot should be brief, and that file system activity should resume after a time no longer than that required for an unmount. This suggests to me that something is wrong with our setup, but I still have no idea what. However, I have found some messages from users with experience similar to ours e.g. http://www.mail-archive.com/freebsd-stable@freebsd.org/msg67320.html Dan Feenberg feenberg isat nber dotte org 617-588-0343 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Making UFS snapshots
I notice on this list that Garance Drosehn http://docs.FreeBSD.org/cgi/mid.cgi?p06230924bf1c752ccf7f reports making a snapshot of a 4 gigabyte filesystem in less than one second. We have a 859 gigabyte filesystem and snapshots take about 75 minutes to complete. Once done they appear to be exactly as advertised. Since we don't yet have any actual files on the filesystem, we anticipated snapshots would be near instantaneous. Even if time were linear in gross filesystem size it should still be done in a minute or so. During this time any other activity referencing (even reads) that filesystem is blocked. Drive activity is continuous all during the 75 minutes, but cpu usage is only a few percent. The filesystem is on 4 300 gigabyte Maxtor SATA drives with a 3ware 9500S-8 controller in raid-5 mode and using the FreeBSD supplied driver. Another poster suggested reducing the number of inodes. Using tunefs -f to increase the average file size from 16K to 64K reduced the time to create a snapshot to 45 minutes. The snapshot size doesn't change. The mdconfig and mounting the memory device take only a fraction of a second - it is only making the snapshot file that takes so long. This is with FreeBSD 5.4 Release #0 (right off the distribution disk, no additional software). Is there likely a problem with our setup, or should we give up on the plan of nightly snapshots? Is this a product of Raid 5, 3ware, super-linearity or to be expected? Thanks Daniel Feenberg National Bureau of Economic Research feenberg isat nber dotte org 617-588-0343 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Making UFS snapshots
At 1:48 AM +0300 8/7/05, Michael Dexter wrote: Hello Garance and all, snip Garance wrote: I think there's a writeup somewhere on making/using snapshots. I'll see if I can remember where it is. Any pointers are appreciated. Seriously, I can't find any useful documentation on how they work or what commands are involved. :( Odd, I see snapshot(8) on the web page but not my 5.4 system. Well, this is something I wrote up for some user's group or another. This was meant as a terse handout that I gave to people, and then I talked about each step in detail at the presentation. I probably should clean it up to make more sense, but hopefully this will give you the basic idea: * Example of making and using a UFS snapshot (as available in FreeBSD 5.3-release and later) See also http://www.mckusick.com/softdep/index.html * # Before the snapshot: df -k Filesystem 1K-blocksUsedAvail Capacity Mounted on /dev/ad4s3a272558 60038 19071624%/ devfs 1 10 100%/dev /dev/ad4s3e 4164558 1315622 251577234%/usr /dev/ad4s3d421358 19344 368306 5%/var cd /usr ls .snap/ cvs/lib/obj/src/ X11R6/ games/ libdata/ports/ bin/home/ libexec/sbin/ compat/ include/local/ share/ * # Creating the snapshot (I use a 'time' command to show # how quickly the snapshot is made, in real time, while # the partition is mounted and a few processes are # actively using it...) /usr/bin/time mount -u -o snapshot /usr/.snap/2004_1006 /usr 0.60 real 0.00 user 0.04 sys ls -l /usr/.snap/2004_1006 -r 1 root operator ... 4404019424 Oct 6 14:13 /usr/.snap/2004_1006 * # Attach that snapshot file to a memory device /usr/bin/time mdconfig -a -t vnode -u 0 \ -f /usr/.snap/2004_1006 0.03 real 0.00 user 0.00 sys * # Mount that memory device for user access mkdir -p /WayBack/usr-2004_1006 chmod 755 /WayBack /WayBack/usr-2004_1006 mount -r /dev/md0 /WayBack/usr-2004_1006 * # After it is mounted: df -k Filesystem 1K-blocksUsedAvail Capacity Mounted on /dev/ad4s3a272558 60038 190716 24%/ devfs 1 10 100%/dev /dev/ad4s3e 4164558 1318198 2513196 34%/usr /dev/ad4s3d421358 19344 3683065%/var /dev/md0 4164558 1315622 2515772 34%/WayBack/usr-2004_1006 cd /WayBack/usr-2004_1006 ls .snap/ cvs/lib/obj/src/ X11R6/ games/ libdata/ports/ bin/home/ libexec/sbin/ compat/ include/local/ share/ * # Getting rid of the snapshot umount /WayBack/usr-2004_1006 mdconfig -d -u 0 rm /usr/.snap/2004_1006 override r root/operator snapshot ... for /usr/.snap/2004_1006? y -- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute; Troy, NY; USA ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
