Ok, so it seems that with DiskSuite, detaching a mirror does nothing to
the disk you detached.
However, "zpool detach" appears to mark the disk as blank, so nothing
will find any pools (import, import -D etc). zdb -l will show labels,
but no amount of work that we have found will bring the HDD back online
in the new server. Grub is blank, and findroot can not see any pool.
zpool will not let you offline the 2nd disk in a mirror. This is
incorrect behaviour.
You can not "cfgadm unconfigure" the sata device while zpool has the disk.
We can just yank the disk, but we had issues getting a new-blank disk
recognised after that. cfgadm would not release the old disk.
However, we found we can do this:
# cfgadm -x sata_port_deactivate sata0/1::dsk/c0t1d0
This will make zpool mark it:
c0t1d0s0 REMOVED 0 0 0
and eventually:
c0t1d0s0 FAULTED 0 0 0 too many errors
After that, we pull out the disk, and issue:
# zpool detach zboot c0t1d0s0
# cfgadm -x sata_port_activate sata0/1::dsk/c0t1d0
# cfgadm -c configure sata0/1::dsk/c0t1d0
# format (fdisk, partition as required to be the same)
# zpool attach zboot c0t0d0s0 c0t1d0s0
There is one final thing to address, when the disk is used in a new
machine, it will generally panic with "pool was used previously with
system-id xxxxxx". Which requires more miniroot work. It would be nice
to be able to avoid this as well. But you can't export the "/" pool
before pulling out the disk, either.
Jorgen Lundman wrote:
Hello list,
Before we started changing to ZFS bootfs, we used DiskSuite mirrored ufs
boot.
Very often, if we needed to grow a cluster by another machine or two, we
would simply clone a running live server. Generally the procedure for
this would be;
1 detach the "2nd" HDD, metaclear, and delete metadb on 2nd disk.
2 mount the 2nd HDD under /mnt, and change system/vfstab to be a single
boot HDD, and no longer "mirrored", as well as host name, and IP addresses.
3 bootadm update-archive -R /mnt
4 unmount, cfgadm unconfigure, and pull out the HDD.
and generally, in about ~4 minutes, we have a new live server in the
cluster.
We tried to do the same thing to day, but with a ZFS bootfs. We did:
1 zpool detach on the "2nd HDD".
2 cfgadm unconfigure the HDD, and pull out the disk.
The source server was fine, could insert new disk, attach it, and it
resilvered.
However, the new destination server had lots of issues. At first, grub
would give no menu at all, just the grub? command prompt.
The command: findroot(pool_zboot,0,a) would return "Error 15: No such
file".
After booting a Solaris Live CD, I could "zpool import" the pool, but of
course it was in Degraded mode etc.
Now it would show menu, but if you boot it, it would flash the message
that the pool was last accessed by Solaris $sysid, and "panic".
After a lot of reboots, and fiddling, I managed to get miniroot to at
least boot, then, only after inserting a new HDD and letting the pool
become completely "good" would it let me boot into multi-user.
Is there something we should do perhaps, that will let the cloning
procedure go smoothly? Should I "export" the 'now separated disk'
somehow? In fact, can I mount that disk to make changes to it before
pulling out the disk?
Most documentation on cloning uses "zfs send", which would be possible,
but 4 minutes is hard to beat when your cluster is under heavy load.
Lund
--
Jorgen Lundman | <lund...@lundman.net>
Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell)
Japan | +81 (0)3 -3375-1767 (home)
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
