Ethan Erchinger wrote: > Hello, > We have a backup strategy that involves mapping LUNs between a given > pair of hosts, and copying data from one of the LUNs (src) and another > LUN (dest). The src LUNs sit a SAN device, sometimes multiple devices > (zpool mirror). The src LUN is running a MySQL database and typically > will be running for weeks without issue. > > When we start the backup sequence, we map a previously unmapped LUN to > the DB host and issue the following commands: > > root# cfgadm -al > (sleep 10) > root# luxadm probe > (sleep 10) > root# zpool import <pool_name> > > After importing we'll perform some minor IO on the dest LUN, such as > adding a symlink, removing some old configuration files. Then we'll > start an ibbackup of that database from the src LUN to the dest LUN, > and things go bad. > > It's not completely consistent, but sometimes the DB host will crash, > sometimes we'll get chksum/read/write errors on the src LUN. Looking > at dmesg (when the host doesn't crash), we see the LUNs paths all > disappear and then reappear usually around 20 seconds later. Example > output below. Each LUN has 2 paths out of the DB host and 4 paths on > each storage device, across two separate SANs. > > Usually the host will crash when not running with a zpool mirror, > which apparently in Sol10u4, it's expected behavior. > > These hosts are x86_64 servers, running Sol10u4, unpatched. They use > qlogic qla2342 HBAs, and the stock qlc driver. They are using MPXIO, > from what I can tell. > > If anyone has any tips on troubleshooting, or knows of things we are > doing wrong, help would be appreciated.
Hey Ethan, Gotta tell you, this is a pretty gutsy and interesting backup solution. :) While I'll give you props for coming up with something innovative I'd strongly recommend against this type of solution. Manipulation like this really is flirting with a panic. Adding and removing devices too frequently on FC is in my experience dangerous in production environments. If you want to stick with a solution on FC I'd recommend looking at AVS Instant Image and/or Remote Mirror (http://www.opensolaris.org/os/project/avs/). That'll give you a proper replication solution. Since your running ZFS, I _highly_ recommend snapshotting the filesystem containing MySQL data and then using zfs send/recv to move the data to a remote host. Since you paid good money for ibbackup, I'm going to assume you like it and have a production database that you want to be gentle too. I'd consider using NFS as a backup destination or providing a backup destintation LUN to the server and just leaving it, perhaps using UFS so that you could mount it elsewhere easily without importing/exporting LUNs on the host. My philosophy with FC on Solaris in a production environment is to set it and step away. If you can utilize ethernet, particularly with ZFS send/recv I think you'll be happier and avoid the cost of ibbackup. I didn't really answer your question, but hopefully there are some options here that might be more appealing. If you want additional detail on any of them please let me know. benr. _______________________________________________ storage-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/storage-discuss
