We're running oVirt 3.6 on 9 Dell Blades but with just two fairly fat fabrics, one for LAN stuff, ovirtmgmt and one for iSCSI to the storage domains.

15 VM Storage Domains
iSCSI has 4 paths going through a 40Gbit i/o blade to switch

115 VMs or thereabouts
9 VLANS, sharing an i/o blade with ovirtmgmt 40Gbit to switch
500+ virtual disks

What we are seeing more and more is that if we do an operation like expose a new LUN and configure a new storage domain, that all of the hyervisors go "red triangle" and "Connecting..." and it takes a very long time (all day) to straighten out.

My guess is that there's too much to look at vdsm wise and so it's waiting a short(er) period of time for a completed response than what vdsm is going to us, and it just cycles over and over until it just happens to work.

I'm thinking that vdsm having DEBUG enabled isn't helping the latency, but as far as I know it came this way be default. Can we safely disable DEBUG on the hypervisor hosts for vdsm? Can we do this while things are roughly in a steady state? Remember, just doing the moves could throw everything into vdsm la-la-land (actually, that might not be true, might take a new storage thing to do that).

Just thinking out loud... can we safely turn off DEBUG logging on the vdsms? Can we do this "live" through bouncing of the vdsm if everything is "steady state"? Do you think this might help the problems we're having with storage operations? (I can see all the blades logging in iSCSI wise, but ovirt engine does the whole red triangle connecting thing, for many, many, many hours).


Users mailing list

Reply via email to