On Sat, May 5, 2012 at 11:38 PM, Shankhadeep Shome <[email protected]>wrote:
> Hi Chris > > We have a solution we are using on Oracle Exalogic hardware (we are using > the bare metal boxes and gateway switches). I think I understand the > requirement, IB accessible storage from VMs is possible however its a bit > convoluted. Our solution was to create a one-to-one NAT from the VMs to the > IB IPoIB network. This allows the VMs to mount storage natively over the IB > network. The performance is pretty good, about 9Gbps per node with 64k MTU > sizes. We created an open nebula driver for this and I'm happy to share it > with the community. The driver handles VM migrations by enabling/diabling > ip aliases on the host and can also be used to manipulate iptable rules on > source and destination when open nebula moves VMs around. > > Shankhadeep > > > On Sat, May 5, 2012 at 5:42 PM, Chris Barry <[email protected]> wrote: > >> Reading more, I see that the available methods for block store are iSCSI, >> and that the LUNS are attached to the host. From there, a symlink tree >> exposes the target to the guest in a predictable way on every host. >> >> So then to modify my question a bit, are all LUNs attached to all hosts >> simultaneously? Or does the attachment only happen when a migration is to >> occur? Also, is the LUN put into a read-only mode or something during >> migration on the original host to protect the data? Or, must a clustering >> filesystem be employed? >> >> Guess I have a lot to read :) >> >> Thanks >> -C >> >> >> On Sat, May 5, 2012 at 5:23 PM, Chris Barry <[email protected]> wrote: >> >>> Hi Gubda, >>> >>> Thank you for replying. My goal was to use a single shared iso image to >>> boot from, and use an in-memory minimal linux on each node that had no >>> 'disk' at all, then to mount logical data volume(s) from a centralized >>> storage system. Perhaps that is outside the scope of opennebula's design >>> goals, and may not be possible - I'm just now investigating it. >>> >>> I see you are using NFS, but my desire is to use block storage instead, >>> ideally LVM, and not incur the performance penalties of IPoIB. It does >>> sound simple though, and that's always good. Do you have any performance >>> data on that setup in terms of IOPs and/or MB/s write speeds? It does sound >>> interesting. >>> >>> Thanks again, >>> -C >>> >>> >>> On Sat, May 5, 2012 at 4:01 PM, Guba Sándor <[email protected]> wrote: >>> >>>> Hi >>>> >>>> I'm using InfiniBand to make shared storage. My setup is simple: the >>>> opennebula installdir is shared on NFS with the worker nodes. >>>> >>>> - I don't understand what you mean on shared image. There will be a >>>> copy (or symlink if the image is persistent) on the NFS host and that is >>>> that the hypervisor will use over the network. Live migrate is available >>>> beacuse you don't move the image only another host will use it from the >>>> same spot. With my linux images I have about 30s delay when livemigrate. >>>> You can use qcow2 driver for shared image. >>>> >>>> - I don't understant exactly what you mean on "guest unaware". If you >>>> mean storage - host connection it has nothing to do with nebula. You can >>>> use any shared filesystem. NFS uses IPoIB connection. >>>> >>>> 2012-05-05 19:01 keltezéssel, Chris Barry írta: >>>> >>>> Greetings, >>>> >>>> I'm interested in hearing user accounts about using infiniband as the >>>> storage interconnect with OpenNebula if anyone has any thoughts to share. >>>> Specifically about: >>>> * using a shared image and live migrating it (e.g. no copying of >>>> images). >>>> * is the guest unaware of the infiniband or is it running IB drivers? >>>> * does the host expose the volumes to the guest, or does the guest >>>> connect directly? >>>> * I'd like to avoid iSCSI over IPoIB if possible. >>>> * clustering filesystem/LVM requirements. >>>> * file-based vdisk or logical volume usage? >>>> >>>> Anything, an experiences at all will be helpful. >>>> >>>> Thanks >>>> Christopher >>>> >>>> >>>> >>>> _______________________________________________ >>>> Users mailing >>>> [email protected]http://lists.opennebula.org/listinfo.cgi/users-opennebula.org >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> [email protected] >>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org >>>> >>>> >>> >> >> _______________________________________________ >> Users mailing list >> [email protected] >> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org >> >> >
_______________________________________________ Users mailing list [email protected] http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
