Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-29 Thread Nicolas Williams
On Sat, Dec 27, 2008 at 02:29:58PM -0800, Ross wrote: All of which sound like good reasons to use send/receive and a 2nd zfs pool instead of mirroring. Yes. Send/receive has the advantage that the receiving filesystem is guaranteed to be in a stable state. How would you go about recovering

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread dick hoogendijk
On Sat, 27 Dec 2008 14:29:58 PST Ross myxi...@googlemail.com wrote: All of which sound like good reasons to use send/receive and a 2nd zfs pool instead of mirroring. Send/receive has the advantage that the receiving filesystem is guaranteed to be in a stable state. Can send/receive be used

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread Volker A. Brandt
Send/receive has the advantage that the receiving filesystem is guaranteed to be in a stable state. Can send/receive be used on a multiuser running server system? Yes. Will this slowdown the services on the server much? Depends. On a modern box with good disk layout it shouldn't.

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread Kees Nuyt
On Sun, 28 Dec 2008 15:27:00 +0100, dick hoogendijk d...@nagual.nl wrote: On Sat, 27 Dec 2008 14:29:58 PST Ross myxi...@googlemail.com wrote: All of which sound like good reasons to use send/receive and a 2nd zfs pool instead of mirroring. Send/receive has the advantage that the receiving

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread Tim
On Sat, Dec 27, 2008 at 3:24 PM, Miles Nordin car...@ivy.net wrote: t == Tim t...@tcsac.net writes: t couldn't you simply do a detach before removing the disk, and t do a re-attach everytime you wanted to re-mirror? no, for two reasons. First, when you detach a disk, ZFS writes

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-27 Thread Miles Nordin
t == Tim t...@tcsac.net writes: t couldn't you simply do a detach before removing the disk, and t do a re-attach everytime you wanted to re-mirror? no, for two reasons. First, when you detach a disk, ZFS writes something to the disk that makes it unrecoverable. The simple-UI

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-27 Thread Ross
All of which sound like good reasons to use send/receive and a 2nd zfs pool instead of mirroring. Send/receive has the advantage that the receiving filesystem is guaranteed to be in a stable state. How would you go about recovering the system in the event of a drive failure though? Would you

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-22 Thread Ross Smith
On Fri, Dec 19, 2008 at 6:47 PM, Richard Elling richard.ell...@sun.com wrote: Ross wrote: Well, I really like the idea of an automatic service to manage send/receives to backup devices, so if you guys don't mind, I'm going to share some other ideas for features I think would be useful.

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-19 Thread Richard Elling
Ross wrote: Well, I really like the idea of an automatic service to manage send/receives to backup devices, so if you guys don't mind, I'm going to share some other ideas for features I think would be useful. cool. One of the first is that you need some kind of capacity management and

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Wed, Dec 17, 2008 at 10:02:18AM -0800, Ross wrote: In fact, thinking about it, could this be more generic than just a USB backup service? Absolutely. The tool shouldn't need to know that the backup disk is accessed via USB, or whatever. The GUI should, however, present devices

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
Absolutely. The tool shouldn't need to know that the backup disk is accessed via USB, or whatever. The GUI should, however, present devices intelligently, not as cXtYdZ! Yup, and that's easily achieved by simply prompting for a user friendly name as devices are attached. Now you could

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Thu, Dec 18, 2008 at 07:05:44PM +, Ross Smith wrote: Absolutely. The tool shouldn't need to know that the backup disk is accessed via USB, or whatever. The GUI should, however, present devices intelligently, not as cXtYdZ! Yup, and that's easily achieved by simply prompting

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
On Thu, Dec 18, 2008 at 7:11 PM, Nicolas Williams nicolas.willi...@sun.com wrote: On Thu, Dec 18, 2008 at 07:05:44PM +, Ross Smith wrote: Absolutely. The tool shouldn't need to know that the backup disk is accessed via USB, or whatever. The GUI should, however, present devices

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Thu, Dec 18, 2008 at 07:55:14PM +, Ross Smith wrote: On Thu, Dec 18, 2008 at 7:11 PM, Nicolas Williams nicolas.willi...@sun.com wrote: I was thinking more something like: - find all disk devices and slices that have ZFS pools on them - show users the devices and pool names (and

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Richard Elling
Nicolas Williams wrote: On Thu, Dec 18, 2008 at 07:55:14PM +, Ross Smith wrote: On Thu, Dec 18, 2008 at 7:11 PM, Nicolas Williams nicolas.willi...@sun.com wrote: I was thinking more something like: - find all disk devices and slices that have ZFS pools on them - show users

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
Of course, you'll need some settings for this so it's not annoying if people don't want to use it. A simple tick box on that pop up dialog allowing people to say don't ask me again would probably do. I would like something better than that. Don't ask me again sucks when much, much later

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
I was thinking more something like: - find all disk devices and slices that have ZFS pools on them - show users the devices and pool names (and UUIDs and device paths in case of conflicts).. I was thinking that device pool names are too variable, you need to be reading serial numbers

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Thu, Dec 18, 2008 at 12:57:54PM -0800, Richard Elling wrote: Nicolas Williams wrote: Device names are, but there's no harm in showing them if there's something else that's less variable. Pool names are not very variable at all. I was thinking of something a little different. Don't

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross
Well, I really like the idea of an automatic service to manage send/receives to backup devices, so if you guys don't mind, I'm going to share some other ideas for features I think would be useful. One of the first is that you need some kind of capacity management and snapshot deletion.

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Ross
Thinking about it, I think Darren is right. An automatic send/receive to the external drive may be preferable, and it sounds like it has many advantages: 1. It's directional, your backups will always go from your live drive to the backup, never the other way unless you actually force it with

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Ross
In fact, thinking about it, could this be more generic than just a USB backup service? If this were a scheduled backup system, regularly sending snapshots to another device, with a nice end user GUI, there's nothing stopping it working with any device the user points it at. So you could use

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Nicolas Williams
On Wed, Dec 17, 2008 at 12:05:50AM -0800, Ross wrote: Thinking about it, I think Darren is right. An automatic send/receive to the external drive may be preferable, and it sounds like it has many advantages: You forgot *the* most important advantage of using send/recv instead of mirroring as

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Niall Power
What serious compat issues ? There has been one and only one incompatible change in the stream format and that only impacted really really early (before S10 FCS IIRC) adopters. Here are the issues that I am aware: - Running zfs upgrade on a zfs filesystem will cause the zfs send stream

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Nicolas Williams
On Wed, Dec 17, 2008 at 08:51:54AM -0800, Niall Power wrote: What serious compat issues ? There has been one and only one incompatible change in the stream format and that only impacted really really early (before S10 FCS IIRC) adopters. Here are the issues that I am aware: -

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Ross
Hey Niall, Here are the issues that I am aware: - Running zfs upgrade on a zfs filesystem will cause the zfs send stream output format to be incompatible with older versions of the software. This is according to the zfs man page. - Again from the zfs man page: The format of the stream is

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Niall Power
In the long run some USB stick problems may surface because the wearbr leveling is done in 16MB sections, and you could blow your stick ifbr you have a 16MB region which is ``hot#39;#39;. nbsp;I wonder if parts of a zpoolbr are hotter than others? nbsp;With AVS the dirty bitmap might be

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Ross
Does 1. really need to be fixed? I ask this since I imagine there will be some resistance from the ZFS team to essentially breaking the spec for the sake of not confusing some users. I would argue that anybody who knows enough to run zpool status is also capable of learning what a mirror is

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
Andrew Gabriel wrote: Different USB memory sticks vary enormously in speed. The speed is often not described on the packaging, so it's often not possible to know how fast one is until after you've bought it and tried it. This was tested with an external laptop hardisk inside a USB

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
Does 1. really need to be fixed? I'm not suggesting that it's currently broken I'm just asking if it would be reasonable to special case our usage a little bit in order to avoid unnecessary alarm to users. This will be seen as a fit and finish/polish issue. If it's easy to address that then

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
Hi Volker, Yes, by all means. I am doing something very similar on my T1000, but I have two separate one-disk pools and copy to the backup pool using rsync. I would very much like to replace this with automatic resilvering. One prerequisite for wide adoption would be to fix the issue

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
Yes to both I believe, while the USB device is attached your system will run slower, and it will run considerably slower while replicating data. Hopefully USB 3 or eSATA drives would address this to some extent. I think I've confirmed this is the case, at least in the configuration I

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Ross
One other question I have about using mirrors is potential performance implications. In a common scenario the user might be using the main S(ATA) attached disk and a USB external disk as a mirror configuration. Could the slower disk become a bottleneck because of it's lower I/O read/write

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Miles Nordin
np == Niall Power niall.po...@sun.com writes: np So I'd like to ask if this is an appropriate use of ZFS mirror np functionality? I like it a lot. I tried to set up something like that ad-hoc using a firewire disk on an Ultra10 at first, and then, just as you thought, tried using one

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Tim
On Tue, Dec 16, 2008 at 1:53 PM, Miles Nordin car...@ivy.net wrote: np == Niall Power niall.po...@sun.com writes: np So I'd like to ask if this is an appropriate use of ZFS mirror np functionality? I like it a lot. I tried to set up something like that ad-hoc using a firewire disk

[zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-15 Thread Niall Power
Hi all, A while back, I posted here about the issues ZFS has with USB hotplugging of ZFS formatted media when we were trying to plan an external media backup solution for time-slider: http://www.opensolaris.org/jive/thread.jspa?messageID=299501 As well as the USB issues in the subject we became

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-15 Thread Volker A. Brandt
A while back, I posted here about the issues ZFS has with USB hotplugging of ZFS formatted media when we were trying to plan an external media backup solution for time-slider: http://www.opensolaris.org/jive/thread.jspa?messageID=299501 [...] There are a few minor issues however which I'd