On Mar 16, 2012, at 7:57 PM, Justin Shepherd wrote: > > > Sent from my iPad > > On Mar 16, 2012, at 12:26, "Pádraig Brady" <p...@draigbrady.com> wrote: > >> On 03/16/2012 04:11 PM, Jay Pipes wrote: >>> Hi Stackers, >>> >>> So, in diagnosing a few things on TryStack yesterday, I ran into an >>> interesting problem with snapshotting that I'm hoping to get some advice on. >>> >>> == The Problem == >>> >> >>> QEMU was unhelpfully returning a vague error message of "error while >>> writing". >> We ran into this problem in our Diablo deployment as well. It definitely needs a more informative error message. >> >>> == Possible Solutions == >>> >>> So, there are a number of solutions that we can work on here, and I'm >>> wondering what the preference would be. Here are the solutions I have come >>> up with, along with a no-brainer improvement to Nova that would help in >>> diagnosing this problem: >>> >>> The no-brainer: Detect before attempting a snapshot that there is enough >>> space on a device to perform the operation, and if not, throw a useful >>> error message up the stack >> >> The space can change while writing, so you could still get the same error >> above. >> >>> >>> Solutions to the disk space problem: >>> >>> (1) Silly Jay, change the damn size of the root partition in your PXE base >>> OS install! >>> >>> Now, I'm no expert in creating customized base disk images, but from >>> looking at the build_pxe_env.sh script in devstack [1], it seems pretty >>> trivial to change the ramdisk_size parameter in the startup options to >>> something larger than 2109600. We could do this and reimage the compute >>> nodes one by one. >>> >>> (2) Make the location in which the snapshot is made configurable. >>> >>> Right now, as mentioned above, tempfile.mkdtemp() is used, which creates a >>> directory in the user's TMPDIR (typically /tmp, which is usually on the >>> root partition). >>> >>> We could add an option (--libvirt-snapshot-dir?) that would allow >>> nova-compute to override where that snapshot is built. >>> >>> (3) Change the user (running nova-compute) TMPDIR setting to something >>> different than /tmp on the root partition). >> >> I'd lean towards (3). >> That's something that depends on the environment (as you've nicely >> demonstrated), >> and also for security reasons the admin should be able to set TMPDIR. >> That's the standard way to do it, and it works already (hopefully). > > Actually I would argue that the best way to accomplish this would be option > #2. That way an admin/operator has control over the location. Not > manipulating this by messing around with a users environment variable. > I agree with Pádraig that option #2 is a better way to go. I'd also recommend that it default to something under /var/lib/nova/instances, since that directory should generally be mounted a large partition by default. Take care, Lorin -- Lorin Hochstein Lead Architect - Cloud Services Nimbis Services, Inc. www.nimbisservices.com
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
