Cool. Let me know if you have questions. My instinct is that we probably want to keep the Ova manipulation in the context of vmware since I don't believe it will be used outside that context. Trying to manipulate the ovf files with generic tools may prove to be more complicated to manage going forward as it is almost guaranteed to require some 'hacks' to make it work. If we can avoid those by using the vim jars, it may be worth it. I have not reviewed anything on the vim jars side, so I don't know how good of an option that is.
Are there key benefits to not using the vim jars? Cheers, Will On May 3, 2017 3:34 AM, "Abhinandan Prateek" < abhinandan.prat...@shapeblue.com> wrote: > Hi Will, > > I am improving the multiple disk OVA feature. As part of revamp I am > moving out some OVF manipulation code from the vmware hypervisor plugin > context to secondary storage component. The existing code was using vim25 > and managed objects to query and rewrite the OVF file. I have rewritten > that, using standard java w3c dom parser. > > The overall flow is mostly similar and as below: > 1. Decompress OVA and read the OVF file. OVF file will give information > about various disks > 3. Create the regular cloudstack template out for the boot disk and > rewrite the OVF file, minus the information about other disks. > 4. For each additional disk create data disk templates and capture the > relationship in db. > 5. This can then be followed by creating the multi-disk cloudstack VM. > > Essentially I am rewriting the original OVF file after removing the File > and Disk information that refers to the other disks. Given that the the > VMWare is picky, I think it will require some more cleanup and massaging. > Your inputs will definitely help. > > Overall I think the two pieces, the tool that you have and the cloudstack > multi disk OVA functionality can nicely complement each other. Will post my > learning here. > > Thanks and regards, > -abhi > > > > > On 02/05/17, 6:05 PM, "williamstev...@gmail.com on behalf of Will > Stevens" <williamstev...@gmail.com on behalf of wstev...@cloudops.com> > wrote: > > >Hey Abhinandan, > >First, can you give us a bit more context regarding what you are doing so > >we can highlight potential areas to watch out for? I have done some OVF > >parsing/modification and there are a bunch of gotchas to be aware of. I > >will try to outline some of the ones I found. I have not tried to use the > >vim25.jar, so I can't really help on that front. > > > >In my use case, I was exporting VMs via the ovftool from a source VMware > >environment, and I was migrating them to an ACS managed VMware > >environment. In doing so, I also wanted to support VMs with multiple > disks > >using a Root volume and multiple Data volumes, as well as change the nic > >type (vmxnet3), assign static IPs, etc... I have not had the time to open > >source my migration tool, but it is on my todo list. > > > >My general flow was: > >- Export the VM with ovftool > >- Extract the resulting OVA into its parts (OVF, VMDKs, Manifest) > >- Duplicate the OVF file, once per VMDK > >- Modify a OVF file to be specific for each of the VMDKs (one OVF per > VMDK) > >- Take each VMDK and the corresponding OVF and recompress them back into > an > >OVA > >- Treat the first OVA as a template and the rest as data disks > > > >My initial (naive) approach was to just treat the OVF as a well behaved > XML > >file and use standard XML libs (in my case in Python) to parse and > >manipulate the OVF file. This approach had a few pitfalls which I will > >outline here. > > > >VMware is VERY picky about the format of the OVF file, if the file is not > >perfect, VMware won't import it (or at least the VM won't launch). There > >were two main items which caused me issues. > > > >a) The <Envelope> tag MUST have all of the namespace definitions even if > >they are not used in the file. This is something that most XML parsers > are > >confused by. Most XML parsers will only include the namespaces used in > the > >file when the file is saved. I had to ensure that the resulting OVF files > >had all of the original namespace definitions for the file to import > >correctly. If I remember correctly, they even had to be in the right > >order. I did this by changing the resulting file after saving it with the > >XML lib. > > > >b) VMware uses namespaces which actually collide with each other. For > >example, both the default namespace and the 'ovf' namespace share the same > >URL. Again, XML libraries don't understand this, so I had to manage that > >manually. Luckily, the way VMware handles these namespaces is relatively > >consistent, so I was able to find a workaround. Basically, the default > >namespace will apply to all of the elements, and the 'ovf' namespace will > >be applied only in the attributes. Because of this I was able to just use > >the 'ovf' namespace and then after exporting the file, I did a find > replace > >from '<ovf:' and '</ovf:' to '<' and '</' respectively. > > > >Those are the main gotchas which I encountered. > > > >I put the OVA Split function I wrote into a Gist [1] (for now) for your > >reference in case reviewing the code is helpful. I was under a lot of > time > >pressure when building this tool, so I have a bunch of cleanup to do > before > >I release it as open source, but I can rush it out and clean it up after > >release if you are solving the same(ish) problem and my code will be > useful. > > > >[1] https://gist.github.com/swill/f6b54762ffcce85772535a490a9c8cbe > > > >I hope this is helpful in your case. > > > >Cheers, > > > >*Will STEVENS* > >Lead Developer > > > ><https://goo.gl/NYZ8KK> > > > >On Tue, May 2, 2017 at 3:49 AM, Abhinandan Prateek < > >abhinandan.prat...@shapeblue.com> wrote: > > > >> Hello, > >> > >> I am looking at vim25.jar to put together ovftool like functionality > >> specially around parsing and generating OVF files. vim25.jar is already > >> included as non-oss dependency and used by vmware hypervisor plugin. I > see > >> that some OVF parsing capabilities are present in this jar, but it > seems to > >> be tied to host connection/context. Can anyone who has used this can > tell > >> me if I can use it as a standalone OVF manipulation api any pointer to > good > >> resource on that will be nice. > >> > >> Regards, > >> -abhi > >> > >> > >> > >> abhinandan.prat...@shapeblue.com > >> www.shapeblue.com > >> 53 Chandos Place, Covent Garden, London WC2N 4HSUK > >> @shapeblue > >> > >> > >> > >> > > abhinandan.prat...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > >
