Hello all, This is not so much a question but rather a "how-to" for posterity. Comments and possible fixes are welcome, though.
I'm toying (for work) with a Solaris 10 VM, and it has a dedicated virtual HDD for data and zones. The template VM had a 20Gb disk, but a particular application needs more. I hoped ZFS autoexpand would do the trick transparently, but it turned out some manual labor is needed. Even though this was a single-disk pool with a dedicated disk, the EFI label and ultimate Solaris partition and slices did not detect the size change over several devfsadm's, cfgadm's, exports, imports, reboots, formats and fdisks - only when I entered some values by hand did things work out. Many solutions just added a bigger disk, mirrored the pool and removed the smaller disk, and were left with a bigger pool. Sometimes for whatever architectural reason, this way is blocked. Several blogs and forums I googled up pointed me in the right direction, so I'd try to overview the solution in one post :) 0) Backup the VM (disk image, SAN LUN, etc.) in question. Note that depending on the VM technology, disk expansion may not be available if the VM has snapshots, so you may have to make backups or snapshots at real storage level. 1) Run "zpool status poolname" to determine component disks, if you don't already know those details: # zpool status pool: pool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 errors: No known data errors pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors So, here I'll work with "pool" which has a dedicated disk. I guess a similar technique should be valid for slice-based pools in MBR partition tables like "rpool" above, corrected for different partition naming scheme. 2) Resize the HDD image size as is appropriate in your external storage (VM hypervisor, SAN, backing zvol, etc.) The pool's component's can't get smaller now, so by resize I mean increase. 3) Have the OS detect new disk size (use reconfiguration reboot, devfsadm, and/or cfgadm): # devfsadm -Cv and enable autoexpansion (didn't help me much, hence the post): # zpool set autoexpand=on pool 4) Run solaris "format" utility and select the needed disk by number, i.e.: # format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 <DEFAULT cyl 2607 alt 2 hd 255 sec 63> /pci@0,0/pci15ad,7a0@15/pci15ad,1976@0/sd@0,0 1. c1t1d0 <VMware-Virtual disk-1.0-60.00GB> /pci@0,0/pci15ad,7a0@15/pci15ad,1976@0/sd@1,0 Specify disk (enter its number): 1 selecting c1t1d0 [disk formatted] /dev/dsk/c1t1d0s0 is part of active ZFS pool pool. Please see zpool(1M). Now, the system already sees the while disk as 60Gb, but when I inspect the partition table (entering "p", "p"), I only see the original 20Gb (~40mln sector) partition size: format> p partition> p Current partition table (original): Total disk sectors available: 41926621 + 16384 (reserved sectors) Part Tag Flag First Sector Size Last Sector 0 usr wm 256 19.99GB 41926621 1 unassigned wm 0 0 0 2 unassigned wm 0 0 0 3 unassigned wm 0 0 0 4 unassigned wm 0 0 0 5 unassigned wm 0 0 0 6 unassigned wm 0 0 0 8 reserved wm 41926622 8.00MB 41943005 partition> q If you skim over this text, please do actually print your partition table and note the ZFS slice (i.e. "0") size and start location in sectors. Perhaps, write them down on paper. 5) So I go into fdisk to edit lower-level partitions on this drive, right from the format utility with the disk in question selected: format> fd Total disk size is 7832 cylinders Cylinder size is 16065 (512 byte) blocks Cylinders Partition Status Type Start End Length % ========= ====== ============ ===== === ====== === 1 EFI 0 2610 2611 33 Here for a whole disk dedicated to ZFS I see a EFI partition that does not consume my whole disk. For a root pool's disk with MBR I'd see this: Total disk size is 2610 cylinders Cylinder size is 16065 (512 byte) blocks Cylinders Partition Status Type Start End Length % ========= ====== ============ ===== === ====== === 1 Active Solaris 1 2609 2609 100 If you apply this technique to rpools, note that the partition type is different (SOLARIS vs EFI), the start cylinders differ (1 for MBR, 0 for EFI), and the bootable partition is Active. 6) The scary part is that I need to remove the partition and slice tables and recreate them starting at the same positions. So in fdisk I press "3" to delete the partition "1", then I press "1" to create a new partition. If I select "EFI", it automatically fills the disk from 0 to end. An MBR-based (Solaris2) partition started at 1 and asked me to enter desired size. For the disk dedicated fully to a pool, I chose "EFI" as it was originally. Now I press "5" to save the new partition table and return to "format". Entering "p","p" I see that the slice sizes remain as they were... Returning to the disk-level menu, I entered "t" for Type: format> t AVAILABLE DRIVE TYPES: 0. Auto configure 1. other Specify disk type (enter its number)[1]: 0 c1t1d0: configured with capacity of 60.00GB <VMware-Virtual disk-1.0-60.00GB> selecting c1t1d0 [disk formatted] /dev/dsk/c1t1d0s0 is part of active ZFS pool pool. Please see zpool(1M). I picked "0", et voila - the partition sizes are reassigned. Too early to celebrate however: the ZFS slice #0 now starts at a wrong position: format> p partition> p Current partition table (default): Total disk sectors available: 125812701 + 16384 (reserved sectors) Part Tag Flag First Sector Size Last Sector 0 usr wm 34 59.99GB 125812701 1 unassigned wm 0 0 0 2 unassigned wm 0 0 0 3 unassigned wm 0 0 0 4 unassigned wm 0 0 0 5 unassigned wm 0 0 0 6 unassigned wm 0 0 0 8 reserved wm 125812702 8.00MB 125829085 Remembering that the original table started this slice at 256, and remembering the new table's last sector value, I mix the two: partition> 0 Part Tag Flag First Sector Size Last Sector 0 usr wm 34 59.99GB 125812701 Enter partition id tag[usr]: Enter partition permission flags[wm]: Enter new starting Sector[34]: 256 Enter partition size[125812668b, 125812923e, 61431mb, 59gb, 0tb]: 125812701e partition> p Current partition table (unnamed): Total disk sectors available: 125812701 + 16384 (reserved sectors) Part Tag Flag First Sector Size Last Sector 0 usr wm 256 59.99GB 125812701 Finally, I can save the changed tables and exit "format": partition> label Ready to label disk, continue? y partition> q format> q 7) Inspecting the pool, and even exporting and importing it and inspecting again, I see that autoexpand did not take place and the pool is still 20Gb in size (dunno why - sol10u10 bug?) :( So I do the manual step: # zpool online -e pool c1t1d0 The "-e" flag marks the component as eligible for expansion. When all pieces of a top-level vdev become larger, the setting takes effect and the pool finally becomes larger: # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 59.9G 441M 59.4G 0% ONLINE - rpool 19.9G 6.91G 13.0G 34% ONLINE - Now I can finally go to my primary quest and install that large piece of software into a zone that lives on "pool"! ;) HTH, //Jim Klimov _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss