Hi Michael, Thank you for sharing your code and for your great feedback. I have created a wiki page [1] to give this thread more visibility.
A quick note for anyone interested in this migration: Michael's code can be merged into 3.3.0_to_3.3.80.rb or 3.3.80_to_3.4.0.rb to skip the db_versioning table edition. Cheers [1] http://wiki.opennebula.org/upgrade34 -- Carlos Martín, MSc Project Engineer OpenNebula - The Open-source Solution for Data Center Virtualization www.OpenNebula.org | [email protected] | @OpenNebula<http://twitter.com/opennebula><[email protected]> On Fri, Apr 20, 2012 at 7:49 AM, Michael Kutzner <[email protected] > wrote: > Hi, > > the following is a longer email, but I believe there are others also > having the need to make a "smooth" upgrade from 3.2.x to 3.4 > > Thanx again to Carlos to show me the right way, so how did we do the > Migration of 3.2.1 to 3.4 having certain active VMs and not the chance to > shut them down. > Some background for the steps described below: > Our old setup was: > - ssh as transfer manager (a little bit modified) > - VM_DIR was /one/vm in 3.2 > - lots of persistent images, no images marked with SAVE_AS > - path to image repository: /one/var/images > > New setup: > - ssh transfer manager > - filesystem datastore 1 (/one/var/datastores/1) > - system datastore 0 (/one/var/datastores/0) > > The migration steps: > 1) Normal upgrade procedure (installation of one 3.4 without onedb upgrade > yet) > 2) as describe by Carlos I commented out lines 36 to 60 in > /usr/lib/one/ruby/onedb/3.3.0_to_3.3.80.rb > (line 35 does not have to be commented out - "header_done=false") > onedb upgrade run, working fine > 3) On each node create a link for each active vm from > /one/var/datastores/1/<VM_ID> -> /one/vm/<VM_ID>/images > --------- <script>--------- > #!/bin/sh > # > DATASTORE_LOCATION=/one/var/datastores > SYSTEM_STORE=$DATASTORE_LOCATION/0 > IMAGE_STORE=$DATASTORE_LOCATION/1 > VM_DIR=/one/vm > > # Create Datastore Directory > test -d $DATASTORE_LOCATION || mkdir $DATASTORE_LOCATION > # Create System Datastore > test -d $SYSTEM_STORE || (echo "creating vm dir $SYSTEM_STORE"; mkdir > $SYSTEM_STORE) > # > # In our setup the datastore 1 is available on each host, but was mounted > to /one/images > # > test -L $IMAGE_STORE || (echo "linking $IMAGE_STORE"; ln -s /one/images > $IMAGE_STORE ) > > > for i in $VM_DIR/*; do > ID=`basename $i` > if [ -d $i/images -a ! -L $SYSTEM_STORE/$ID -a ! -d $SYSTEM_STORE/$ID ]; > then > echo "linking running vm $ID to new system datastore" > ln -s $i/images $SYSTEM_STORE/$ID > fi > done > ------------------------------ > > After this step ONE was able to monitor again the running VMs, but as I > noticed at the first test, it was not to shutdown the vm and migrate the > images back to the image datastore 1. > What was the problem? > Well the information of the TM_MAD and the datastores was missing in > the persistent images - this information is stored in the the XML body in > the one database, table vm_pool. > <DATASTORE>default</DATASTORE> > <DATASTORE_ID>1</DATASTORE> > <TM_MAD>ssh</TM_MAD> > This information is needed to use the correct transfer manager for the > persistent image migration. > (This creates an error like: > Error executing image transfer script: Transfer Driver > 'hostname:/one/var/datastores/0/1322/disk.0' not available) > > So we had to add the needed information to each persistent disk on > each active vm. > > Diving into ruby for the first time in my life I decided to do > this creating a kind of db update script in for "onedb" way as this brings > out of the box every thing needed (mainly just some data changes in the > database) and was the fastest way for me. > What had to be done? > I created an update script as attached to this email, which does the > following: > - Adds the above mentioned informations to each image, if it is persistent > and not a vm started in the new way (starting at line 46). > - Changes the SOURCE of the images to the correct new location. > > This worked fine for us using again the normal "onedb upgrade", but to > start oned again - now db_version is set to 3.4.0.5 I had to delete the > last entry from the table db_versioning for this version. > (just do a select * from db_versioning, note the oid and delete the row > with the oid having the > entry for version 3.4.0.5) > > After having done this ONE was running fine and shutting down some > "older" test vms worked also fine - migrating back the persistent images > to the datastore 1. > > Beware that when shutting down the "old" vms, only the links on the > nodes are deleted, not the images themselves. This may be done manually > or by a small extension to the delete script of the ssh transfer driver. > > Perhaps this mail gives some info to people having the same need for the > upgrade. > > > Best, Michael > > PS: I like the new version, great work > > [siehe angehängte Datei: 3.4.0_to_3.4.0.5.rb] > > > Am 12.04.2012 um 16:26 schrieb Carlos Martín Sánchez: > > Hi Michael, > > The reason why VMs have to be shut down is because the storage model has > changed quite a bit in this last version, and we couldn't come up with a > migration process that felt solid enough for any deployment. > > But, since you said you are ready for lots of manual work, let's try this: > > First, edit /usr/lib/one/ruby/onedb/3.3.0_to_3.3.80.rb and comment out > lines 35 to 60 [1] to disable the VM status check. > Then upgrade the opennebula DB with the onedb command [2] > > Before you start opennebula again, you need to adapt the previous VM > storage directories to the new 3.4 hierarchy. Assuming you don't have > VM_DIR nor DATASTORE_LOCATION defined in oned.conf, this is how the VM > directories look in both versions: > > 3.2: > /var/lib/one/<vm_id>/images/disk.i > /var/lib/images/abcde > > 3.4: > /var/lib/one/datastores/0/<vm_id>/disk.i > /var/lib/one/datastores/1/abcde > > So what you need to do in the front-end *and each host* is to individually > link each existing VM dir: > > ln -s /var/lib/one/<vm_id>/images /var/lib/one/datastores/0/<vm_id> > > Let me stress that this must be done in all the hosts. > > Please take into account that this is just the basic idea, and that *we > haven't tested this*. You may run into problems depending on your > infrastructure configuration. For instance, if you were using the tm_shared > drivers, VMs containing persistent Images will have links like this one: > > /var/lib/one/7/images/disk.1 -> /var/lib/images/qwerty > > If you move the image files to their new destination in > /var/lib/one/datastores/1, like the documentation says, you will break > those VM disk links. > > > Another problem that may be important to you is that VMs over which you > executed saveas [3] won't be able to save the changes back to the locked > Image once the VM is shut down. In the best scenario, you will find the > files inside /var/lib/one/<vm_id>/images/, but I'm not sure about this, you > may lose your changes... > You can locate these VMs looking for VM/DISK/SAVE_AS, and the Images all > have "-" as the value of IMAGE/SOURCE. > > > As I said, this is not a tested or recommended procedure. > > > Good luck, > Carlos > > [1] > http://dev.opennebula.org/projects/opennebula/repository/revisions/one-3.4/entry/src/onedb/3.3.0_to_3.3.80.rb > [2] http://opennebula.org/documentation:rel3.4:upgrade > [3] > http://opennebula.org/documentation:archives:rel3.2:img_guide#save_changes > > -- > Carlos Martín, MSc > Project Engineer > OpenNebula - The Open-source Solution for Data Center Virtualization > www.OpenNebula.org | [email protected] | > @OpenNebula<http://twitter.com/opennebula><[email protected]> > > > > On Thu, Apr 12, 2012 at 2:41 PM, Michael Kutzner < > [email protected]> wrote: > >> Hello, >> >> first of all congratulations to the final 3.4 and many thanx for the hard >> work! >> >> I just wanted to upgrade to 3.4 and started reading. >> I noticed reading the upgrade guide that all active VMs have to be shut >> down. >> >> Is there any chance to upgrade without powering down all VMs? Even with >> lots of >> manual work? >> >> >> Many thanx, >> Michael >> >> >> _______________________________________________ >> Users mailing list >> [email protected] >> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org >> > > > > > > > _______________________________________________ > Users mailing list > [email protected] > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > >
_______________________________________________ Users mailing list [email protected] http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
