On 08/12/12 05:48, Jordi Espasa Clofent wrote:
Hi all,

The most of my production machines looks like:

* Solaris 10 Update 7 (now we're starting to migrate to Update 10)
* All the FS in classical UFS but /opt with ZFS
* All the zones inside /opt/zones
* All the zones containing a app server (Glassfish)
* All the critical app data is just managed/stored by a backend bbdd, so no data inside the zone but the app (Java files) itself

In case I have some real and weird problem in the zone (or even in the global one) we proceed:

- re-create the zone in another server (all the process is automated by backend scripts, it takes just 15/20 minutes)
Zone cloning can take this down to a few seconds, especially if /opt/zones is a separate zfs file system from /opt. This will make it so that each zone gets its own file system (aka dataset). This is the key to making zone cloning go quickly. Once you have that setup, you can create a master zone that is configured as far as it makes sense in your situation. That may or may not include the glassfish installation and/or the J2EE application(s) being run by glassfish. Then, when you need to create a new zone (or recreate another one that is damaged):

zoneadm -z badzone uninstall -F
zoneadm -z badzone clone master
cp /mumble/badzone.sysidcfg /opt/zones/badzone/root/etc/sysidcfg
chmod 400 /opt/zones/badzone/root/etc/sysidcfg
zoneadm -z badzone boot

Depending on your environment, there may be other first-boot zone-specific setup that is required. You could extend this scheme so that each zone has a custom master that is never subjected to your workload. If something catastrophic happens to the zone that has the workload, you can always uninstall it then re-install from its master.

To make it so that this is easy to do on a recovery system, you can make a copy of your master zone on some other system.

primary# zoneadm -z master detach
primary# zp=`zoneadm list -H -o name /opt/zones/master`
primary# zfs snapshot $zp@replicate
primary# zfs send -p $zp@replicate | ssh backup zfs recv $zp
primary# zoneadm -z master attach

backup# zoneadm -z master create -a /opt/zones/master
backup# zoneadm -z master attach

Depending on your situation, you may or may not want to repeat this with your other zones. Whenever you patch, install new packages in the global zone, or otherwise significantly change the content in a zone that was previously copied, you probably want to redo the copy.

This works best if the packages and patches between primary and backup are exactly in sync. That allows you to move back to primary as well.

- re-deploy the app in the zone (30/45 minutes)

So, in total, if zoneX is crashing, to be able to be up un running again is gonna take around 45min-1h05 aprox. That's acceptable for us, but obviously it would be amazing if we can reduce this needed time.

I'm just wondering if I can do it playing with some ZFS capability (as snapshots) since all the zones are always under /opt/zones, which, as I said, is ZFS.

Thanks in advance for all the suggestions.
zones-discuss mailing list

I assume that you are planning for horrible what-if scenarios. If you are actually experiencing somewhat frequent issues that lead to you needing to rebuild the zones from scratch, you should really figure out the root cause. It could be that you have more serious problems than just the occasional misbehaving zone.

Mike Gerdts
Solaris Core OS / Zones                 http://blogs.oracle.com/zoneszone/

zones-discuss mailing list

Reply via email to