Well, now that I’ve gone and read through that bug again in detail, I’m not 
sure I’ve worked around it after all. I do seem to recall additional discussion 
on the original bug for HA engine ligfapi a mention that RR-DNS would work to 
resolve the issue, but can’t remember the bug ID at the moment. I will test 
thoroughly the next time I update my glusterfs servers. But I firmly believe 
that I’ve never encountered that issue in over 3 years of running gluster with 
libgfapi enabled.. 

I use round robin DNS, and in theory, QEMU retries until it gets a working 
server. I also have said DNS setup in host files on all my hosts and gluster 
servers, having discovered the hard way that when your DNS server runs on an 
ovirt managed VM, you have a bootstrap problem when thing break badly :) 
Somewhere around gluster 3.12, I added backup servers to the mount options for 
my gluster storage volumes as well, and have’t had any issues with that.

And to be frank, the significant performance bonus from libgfapi is still 
absolutely worth it to me even if it means automatic HA won’t work if one 
particular server is down. I can always intervene in the DNS on my hosts if I 
have to, and it just hasn’t come up yet. 

  -Darrell


> On Feb 13, 2020, at 5:19 PM, Strahil Nikolov <hunter86...@yahoo.com> wrote:
> 
> On February 13, 2020 11:51:41 PM GMT+02:00, Stephen Panicho 
> <s.pani...@gmail.com <mailto:s.pani...@gmail.com>> wrote:
>> Darrell, would you care to elaborate on your HA workaround?
>> 
>> As far as I understand, only the primary Gluster host is visible to
>> libvirt
>> when using gfapi, so if that host goes down, all VMs break. I imagine
>> you're using a round-robin DNS entry for the primary Gluster host, but
>> I'd
>> like to be sure.
>> 
>> On Wed, Feb 12, 2020 at 11:01 AM Darrell Budic <bu...@onholyground.com>
>> wrote:
>> 
>>> Yes. I’m using libgfapi access on gluster 6.7 with overt 4.3.8 just
>> fine,
>>> but I don’t use snapshots. You can work around the HA issue with DNS
>> and
>>> backup server entries on the storage domain as well. Worth it to me
>> for the
>>> performance, YMMV.
>>> 
>>> On Feb 12, 2020, at 8:04 AM, Jayme <jay...@gmail.com> wrote:
>>> 
>>> From my understanding it's not a default option but many users are
>> still
>>> using libgfapi successfully. I'm not sure about its status in the
>> latest
>>> 4.3.8 release but I know it is/was working for people in previous
>> versions.
>>> The libgfapi bugs affect HA and snapshots (on 3 way replica HCI) but
>> it
>>> should still be working otherwise, unless like I said something
>> changed in
>>> more recent releases of oVirt.
>>> 
>>> On Wed, Feb 12, 2020 at 9:43 AM Guillaume Pavese <
>>> guillaume.pav...@interactiv-group.com> wrote:
>>> 
>>>> Libgfapi is not supported because of an old bug in qemu. That qemu
>> bug is
>>>> slowly getting fixed, but the bugs about Libgfapi support in ovirt
>> have
>>>> since been closed as WONTFIX and DEFERRED
>>>> 
>>>> See :
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1465810
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1484660
>>>> <https://bugzilla.redhat.com/show_bug.cgi?id=1484227 
>>>> <https://bugzilla.redhat.com/show_bug.cgi?id=1484227>> : "No plans to
>>>> enable libgfapi in RHHI-V for now. Closing this bug"
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1484227 
>>>> <https://bugzilla.redhat.com/show_bug.cgi?id=1484227> : "No plans to
>>>> enable libgfapi in RHHI-V for now. Closing this bug"
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1633642 
>>>> <https://bugzilla.redhat.com/show_bug.cgi?id=1633642> : "Closing this
>> as
>>>> no action taken from long back.Please reopen if required."
>>>> 
>>>> Would be nice if someone could reopen the closed bugs so this
>> feature
>>>> doesn't get forgotten
>>>> 
>>>> Guillaume Pavese
>>>> Ingénieur Système et Réseau
>>>> Interactiv-Group
>>>> 
>>>> 
>>>> On Tue, Feb 11, 2020 at 9:58 AM Stephen Panicho
>> <s.pani...@gmail.com <mailto:s.pani...@gmail.com>>
>>>> wrote:
>>>> 
>>>>> I used the cockpit-based hc setup and "option
>> rpc-auth-allow-insecure"
>>>>> is absent from /etc/glusterfs/glusterd.vol.
>>>>> 
>>>>> I'm going to redo the cluster this week and report back. Thanks for
>> the
>>>>> tip!
>>>>> 
>>>>> On Mon, Feb 10, 2020 at 6:01 PM Darrell Budic
>> <bu...@onholyground.com <mailto:bu...@onholyground.com>>
>>>>> wrote:
>>>>> 
>>>>>> The hosts will still mount the volume via FUSE, but you might
>> double
>>>>>> check you set the storage up as Gluster and not NFS.
>>>>>> 
>>>>>> Then gluster used to need some config in glusterd.vol to set
>>>>>> 
>>>>>>    option rpc-auth-allow-insecure on
>>>>>> 
>>>>>> I’m not sure if that got added to a hyper converged setup or not,
>> but
>>>>>> I’d check it.
>>>>>> 
>>>>>> On Feb 10, 2020, at 4:41 PM, Stephen Panicho <s.pani...@gmail.com 
>>>>>> <mailto:s.pani...@gmail.com>>
>>>>>> wrote:
>>>>>> 
>>>>>> No, this was a relatively new cluster-- only a couple days old.
>> Just a
>>>>>> handful of VMs including the engine.
>>>>>> 
>>>>>> On Mon, Feb 10, 2020 at 5:26 PM Jayme <jay...@gmail.com 
>>>>>> <mailto:jay...@gmail.com>> wrote:
>>>>>> 
>>>>>>> Curious do the vms have active snapshots?
>>>>>>> 
>>>>>>> On Mon, Feb 10, 2020 at 5:59 PM <s.pani...@gmail.com 
>>>>>>> <mailto:s.pani...@gmail.com>> wrote:
>>>>>>> 
>>>>>>>> Hello, all. I have a 3-node Hyperconverged oVirt 4.3.8 cluster
>>>>>>>> running on CentOS 7.7 hosts. I was investigating poor Gluster
>> performance
>>>>>>>> and heard about libgfapi, so I thought I'd give it a shot.
>> Looking through
>>>>>>>> the documentation, followed by lots of threads and BZ reports,
>> I've done
>>>>>>>> the following to enable it:
>>>>>>>> 
>>>>>>>> First, I shut down all VMs except the engine. Then...
>>>>>>>> 
>>>>>>>> On the hosts:
>>>>>>>> 1. setsebool -P virt_use_glusterfs on
>>>>>>>> 2. dynamic_ownership=0 in /etc/libvirt/qemu.conf
>>>>>>>> 
>>>>>>>> On the engine VM:
>>>>>>>> 1. engine-config -s LibgfApiSupported=true --cver=4.3
>>>>>>>> 2. systemctl restart ovirt-engine
>>>>>>>> 
>>>>>>>> VMs now fail to launch. Am I doing this correctly? I should also
>> note
>>>>>>>> that the hosts still have the Gluster domain mounted via FUSE.
>>>>>>>> 
>>>>>>>> Here's a relevant bit from engine.log:
>>>>>>>> 
>>>>>>>> 2020-02-06T16:38:32.573511Z qemu-kvm: -drive file=gluster://
>>>>>>>> 
>> node1.fs.trashnet.xyz:24007/vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78,file.debug=4,format=qcow2,if=none,id=drive-ua
>>  
>> <http://node1.fs.trashnet.xyz:24007/vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78,file.debug=4,format=qcow2,if=none,id=drive-ua>-a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,serial=a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,werror=stop,rerror=stop,cache=none,discard=unmap,aio=native:
>>>>>>>> Could not read qcow2 header: Invalid argument.
>>>>>>>> 
>>>>>>>> The full engine.log from one of the attempts:
>>>>>>>> 
>>>>>>>> 2020-02-06 16:38:24,909Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
>>>>>>>> (ForkJoinPool-1-worker-12) [] add VM
>>>>>>>> 'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'(yumcache) to rerun
>> treatment
>>>>>>>> 2020-02-06 16:38:25,010Z ERROR
>>>>>>>> [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring]
>>>>>>>> (ForkJoinPool-1-worker-12) [] Rerun VM
>>>>>>>> 'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'. Called from VDS '
>>>>>>>> node2.ovirt.trashnet.xyz <http://node2.ovirt.trashnet.xyz/>'
>>>>>>>> 2020-02-06 16:38:25,091Z WARN
>>>>>>>> 
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] EVENT_ID:
>>>>>>>> USER_INITIATED_RUN_VM_FAILED(151), Failed to run VM yumcache on
>> Host
>>>>>>>> node2.ovirt.trashnet.xyz <http://node2.ovirt.trashnet.xyz/>.
>>>>>>>> 2020-02-06 16:38:25,166Z INFO
>>>>>>>> [org.ovirt.engine.core.bll.RunVmCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] Lock Acquired to
>> object
>>>>>>>> 
>> 'EngineLock:{exclusiveLocks='[df9dbac4-35c0-40ee-acd4-a1cfc959aa8b=VM]',
>>>>>>>> sharedLocks=''}'
>>>>>>>> 2020-02-06 16:38:25,179Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] START,
>>>>>>>> IsVmDuringInitiatingVDSCommand(
>>>>>>>> 
>> IsVmDuringInitiatingVDSCommandParameters:{vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'}),
>>>>>>>> log id: 2107f52a
>>>>>>>> 2020-02-06 16:38:25,181Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
>>>>>>>> IsVmDuringInitiatingVDSCommand, return: false, log id: 2107f52a
>>>>>>>> 2020-02-06 16:38:25,298Z INFO
>>>>>>>> [org.ovirt.engine.core.bll.RunVmCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] Running command:
>>>>>>>> RunVmCommand internal: false. Entities affected :  ID:
>>>>>>>> df9dbac4-35c0-40ee-acd4-a1cfc959aa8b Type: VMAction group RUN_VM
>> with role
>>>>>>>> type USER
>>>>>>>> 2020-02-06 16:38:25,313Z INFO
>>>>>>>> [org.ovirt.engine.core.bll.utils.EmulatedMachineUtils]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] Emulated machine
>>>>>>>> 'pc-q35-rhel7.6.0' which is different than that of the cluster
>> is set for
>>>>>>>> 'yumcache'(df9dbac4-35c0-40ee-acd4-a1cfc959aa8b)
>>>>>>>> 2020-02-06 16:38:25,382Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] START,
>>>>>>>> UpdateVmDynamicDataVDSCommand(
>>>>>>>> UpdateVmDynamicDataVDSCommandParameters:{hostId='null',
>>>>>>>> vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b',
>>>>>>>> 
>> vmDynamic='org.ovirt.engine.core.common.businessentities.VmDynamic@9774a64'}),
>>>>>>>> log id: 4a83911f
>>>>>>>> 2020-02-06 16:38:25,417Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
>>>>>>>> UpdateVmDynamicDataVDSCommand, return: , log id: 4a83911f
>>>>>>>> 2020-02-06 16:38:25,418Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] START,
>> CreateVDSCommand(
>>>>>>>> 
>> CreateVDSCommandParameters:{hostId='c3465ca2-395e-4c0c-b72e-b5b7153df452',
>>>>>>>> vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b', vm='VM
>> [yumcache]'}), log id:
>>>>>>>> 5e07ba66
>>>>>>>> 2020-02-06 16:38:25,420Z INFO
>>>>>>>> 
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] START,
>>>>>>>> CreateBrokerVDSCommand(HostName = node1.ovirt.trashnet.xyz 
>>>>>>>> <http://node1.ovirt.trashnet.xyz/>,
>>>>>>>> 
>> CreateVDSCommandParameters:{hostId='c3465ca2-395e-4c0c-b72e-b5b7153df452',
>>>>>>>> vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b', vm='VM
>> [yumcache]'}), log id:
>>>>>>>> 1bfa03c4
>>>>>>>> 2020-02-06 16:38:25,424Z INFO
>>>>>>>> 
>> [org.ovirt.engine.core.vdsbroker.builder.vminfo.VmInfoBuildUtils]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] Kernel FIPS -
>> Guid:
>>>>>>>> c3465ca2-395e-4c0c-b72e-b5b7153df452 fips: false
>>>>>>>> 2020-02-06 16:38:25,435Z INFO
>>>>>>>> 
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] VM <?xml
>> version="1.0"
>>>>>>>> encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="
>>>>>>>> http://ovirt.org/vm/tune/1.0 <http://ovirt.org/vm/tune/1.0>"
>> xmlns:ovirt-vm="http://ovirt.org/vm/1.0 <http://ovirt.org/vm/1.0>
>>>>>>>> ">
>>>>>>>>  <name>yumcache</name>
>>>>>>>>  <uuid>df9dbac4-35c0-40ee-acd4-a1cfc959aa8b</uuid>
>>>>>>>>  <memory>1048576</memory>
>>>>>>>>  <currentMemory>1048576</currentMemory>
>>>>>>>>  <iothreads>1</iothreads>
>>>>>>>>  <maxMemory slots="16">4194304</maxMemory>
>>>>>>>>  <vcpu current="1">16</vcpu>
>>>>>>>>  <sysinfo type="smbios">
>>>>>>>>    <system>
>>>>>>>>      <entry name="manufacturer">oVirt</entry>
>>>>>>>>      <entry name="product">OS-NAME:</entry>
>>>>>>>>      <entry name="version">OS-VERSION:</entry>
>>>>>>>>      <entry name="serial">HOST-SERIAL:</entry>
>>>>>>>>      <entry
>> name="uuid">df9dbac4-35c0-40ee-acd4-a1cfc959aa8b</entry>
>>>>>>>>    </system>
>>>>>>>>  </sysinfo>
>>>>>>>>  <clock offset="variable" adjustment="0">
>>>>>>>>    <timer name="rtc" tickpolicy="catchup"/>
>>>>>>>>    <timer name="pit" tickpolicy="delay"/>
>>>>>>>>    <timer name="hpet" present="no"/>
>>>>>>>>  </clock>
>>>>>>>>  <features>
>>>>>>>>    <acpi/>
>>>>>>>>  </features>
>>>>>>>>  <cpu match="exact">
>>>>>>>>    <model>EPYC</model>
>>>>>>>>    <feature name="ibpb" policy="require"/>
>>>>>>>>    <feature name="virt-ssbd" policy="require"/>
>>>>>>>>    <topology cores="1" threads="1" sockets="16"/>
>>>>>>>>    <numa>
>>>>>>>>      <cell id="0" cpus="0" memory="1048576"/>
>>>>>>>>    </numa>
>>>>>>>>  </cpu>
>>>>>>>>  <cputune/>
>>>>>>>>  <devices>
>>>>>>>>    <input type="tablet" bus="usb"/>
>>>>>>>>    <channel type="unix">
>>>>>>>>      <target type="virtio" name="ovirt-guest-agent.0"/>
>>>>>>>>      <source mode="bind"
>>>>>>>> 
>> path="/var/lib/libvirt/qemu/channels/df9dbac4-35c0-40ee-acd4-a1cfc959aa8b.ovirt-guest-agent.0"/>
>>>>>>>>    </channel>
>>>>>>>>    <channel type="unix">
>>>>>>>>      <target type="virtio" name="org.qemu.guest_agent.0"/>
>>>>>>>>      <source mode="bind"
>>>>>>>> 
>> path="/var/lib/libvirt/qemu/channels/df9dbac4-35c0-40ee-acd4-a1cfc959aa8b.org.qemu.guest_agent.0"/>
>>>>>>>>    </channel>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="1">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x0"
>> slot="0x02"
>>>>>>>> type="pci" multifunction="on"/>
>>>>>>>>    </controller>
>>>>>>>>    <memballoon model="virtio">
>>>>>>>>      <stats period="5"/>
>>>>>>>>      <alias name="ua-27c77007-3a3c-4431-958d-90fd1c7257dd"/>
>>>>>>>>      <address bus="0x05" domain="0x0000" function="0x0"
>> slot="0x00"
>>>>>>>> type="pci"/>
>>>>>>>>    </memballoon>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="2">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x1"
>> slot="0x02"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="9">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x0"
>> slot="0x03"
>>>>>>>> type="pci" multifunction="on"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="sata" index="0">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x2"
>> slot="0x1f"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <rng model="virtio">
>>>>>>>>      <backend model="random">/dev/urandom</backend>
>>>>>>>>      <alias name="ua-51960005-6b95-47e9-82a7-67d5e0d6cf8a"/>
>>>>>>>>    </rng>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="6">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x5"
>> slot="0x02"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="15">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x6"
>> slot="0x03"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="13">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x4"
>> slot="0x03"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="7">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x6"
>> slot="0x02"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <graphics type="vnc" port="-1" autoport="yes" passwd="*****"
>>>>>>>> passwdValidTo="1970-01-01T00:00:01" keymap="en-us">
>>>>>>>>      <listen type="network" network="vdsm-ovirtmgmt"/>
>>>>>>>>    </graphics>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="16">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x7"
>> slot="0x03"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="12">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x3"
>> slot="0x03"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <video>
>>>>>>>>      <model type="qxl" vram="32768" heads="1" ram="65536"
>>>>>>>> vgamem="16384"/>
>>>>>>>>      <alias name="ua-8a295e96-40c3-44de-a3b0-1c4a685a5473"/>
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x0"
>> slot="0x01"
>>>>>>>> type="pci"/>
>>>>>>>>    </video>
>>>>>>>>    <graphics type="spice" port="-1" autoport="yes"
>> passwd="*****"
>>>>>>>> passwdValidTo="1970-01-01T00:00:01" tlsPort="-1">
>>>>>>>>      <channel name="main" mode="secure"/>
>>>>>>>>      <channel name="inputs" mode="secure"/>
>>>>>>>>      <channel name="cursor" mode="secure"/>
>>>>>>>>      <channel name="playback" mode="secure"/>
>>>>>>>>      <channel name="record" mode="secure"/>
>>>>>>>>      <channel name="display" mode="secure"/>
>>>>>>>>      <channel name="smartcard" mode="secure"/>
>>>>>>>>      <channel name="usbredir" mode="secure"/>
>>>>>>>>      <listen type="network" network="vdsm-ovirtmgmt"/>
>>>>>>>>    </graphics>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="5">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x4"
>> slot="0x02"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="usb" model="qemu-xhci" index="0"
>> ports="8">
>>>>>>>>      <address bus="0x02" domain="0x0000" function="0x0"
>> slot="0x00"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="4">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x3"
>> slot="0x02"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="3">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x2"
>> slot="0x02"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="11">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x2"
>> slot="0x03"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="scsi" model="virtio-scsi" index="0">
>>>>>>>>      <driver iothread="1"/>
>>>>>>>>      <alias name="ua-d0bf6fcd-7aa2-4658-b7cc-3dac259b7ad2"/>
>>>>>>>>      <address bus="0x03" domain="0x0000" function="0x0"
>> slot="0x00"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="8">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x7"
>> slot="0x02"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="14">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x5"
>> slot="0x03"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="pci" model="pcie-root-port" index="10">
>>>>>>>>      <address bus="0x00" domain="0x0000" function="0x1"
>> slot="0x03"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <controller type="virtio-serial" index="0" ports="16">
>>>>>>>>      <address bus="0x04" domain="0x0000" function="0x0"
>> slot="0x00"
>>>>>>>> type="pci"/>
>>>>>>>>    </controller>
>>>>>>>>    <channel type="spicevmc">
>>>>>>>>      <target type="virtio" name="com.redhat.spice.0"/>
>>>>>>>>    </channel>
>>>>>>>>    <controller type="pci" model="pcie-root"/>
>>>>>>>>    <interface type="bridge">
>>>>>>>>      <model type="virtio"/>
>>>>>>>>      <link state="up"/>
>>>>>>>>      <source bridge="vmnet"/>
>>>>>>>>      <alias name="ua-ceda0ef6-9139-4e5c-8840-86fe344ecbd3"/>
>>>>>>>>      <address bus="0x01" domain="0x0000" function="0x0"
>> slot="0x00"
>>>>>>>> type="pci"/>
>>>>>>>>      <mac address="56:6f:91:b9:00:05"/>
>>>>>>>>      <mtu size="1500"/>
>>>>>>>>      <filterref filter="vdsm-no-mac-spoofing"/>
>>>>>>>>      <bandwidth/>
>>>>>>>>    </interface>
>>>>>>>>    <disk type="file" device="cdrom" snapshot="no">
>>>>>>>>      <driver name="qemu" type="raw" error_policy="report"/>
>>>>>>>>      <source file="" startupPolicy="optional">
>>>>>>>>        <seclabel model="dac" type="none" relabel="no"/>
>>>>>>>>      </source>
>>>>>>>>      <target dev="sdc" bus="sata"/>
>>>>>>>>      <readonly/>
>>>>>>>>      <alias name="ua-bdf99844-2d02-411b-90bb-671ee26764cb"/>
>>>>>>>>      <address bus="0" controller="0" unit="2" type="drive"
>>>>>>>> target="0"/>
>>>>>>>>    </disk>
>>>>>>>>    <disk snapshot="no" type="network" device="disk">
>>>>>>>>      <target dev="sda" bus="scsi"/>
>>>>>>>>      <source protocol="gluster"
>>>>>>>> 
>> name="vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78">
>>>>>>>>        <host name="node1.fs.trashnet.xyz 
>>>>>>>> <http://node1.fs.trashnet.xyz/>" port="0"/>
>>>>>>>>        <seclabel model="dac" type="none" relabel="no"/>
>>>>>>>>      </source>
>>>>>>>>      <driver name="qemu" discard="unmap" io="native"
>> type="qcow2"
>>>>>>>> error_policy="stop" cache="none"/>
>>>>>>>>      <alias name="ua-a1d56b14-6d72-4f46-a0aa-eb0870c36bc4"/>
>>>>>>>>      <address bus="0" controller="0" unit="0" type="drive"
>>>>>>>> target="0"/>
>>>>>>>>      <boot order="1"/>
>>>>>>>>      <serial>a1d56b14-6d72-4f46-a0aa-eb0870c36bc4</serial>
>>>>>>>>    </disk>
>>>>>>>>    <lease>
>>>>>>>>      <key>df9dbac4-35c0-40ee-acd4-a1cfc959aa8b</key>
>>>>>>>> 
>> <lockspace>781717e5-1cff-43a1-b586-9941503544e8</lockspace>
>>>>>>>>      <target offset="6291456"
>> path="/rhev/data-center/mnt/glusterSD/
>>>>>>>> node1.fs.trashnet.xyz <http://node1.fs.trashnet.xyz/>
>>>>>>>> :_vmstore/781717e5-1cff-43a1-b586-9941503544e8/dom_md/xleases"/>
>>>>>>>>    </lease>
>>>>>>>>  </devices>
>>>>>>>>  <pm>
>>>>>>>>    <suspend-to-disk enabled="no"/>
>>>>>>>>    <suspend-to-mem enabled="no"/>
>>>>>>>>  </pm>
>>>>>>>>  <os>
>>>>>>>>    <type arch="x86_64" machine="pc-q35-rhel7.6.0">hvm</type>
>>>>>>>>    <smbios mode="sysinfo"/>
>>>>>>>>  </os>
>>>>>>>>  <metadata>
>>>>>>>>    <ovirt-tune:qos/>
>>>>>>>>    <ovirt-vm:vm>
>>>>>>>>      <ovirt-vm:minGuaranteedMemoryMb
>>>>>>>> type="int">512</ovirt-vm:minGuaranteedMemoryMb>
>>>>>>>>      <ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion>
>>>>>>>>      <ovirt-vm:custom/>
>>>>>>>>      <ovirt-vm:device mac_address="56:6f:91:b9:00:05">
>>>>>>>>        <ovirt-vm:custom/>
>>>>>>>>      </ovirt-vm:device>
>>>>>>>>      <ovirt-vm:device devtype="disk" name="sda">
>>>>>>>> 
>>>>>>>> 
>> <ovirt-vm:poolID>2ffaec76-462c-11ea-b155-00163e512202</ovirt-vm:poolID>
>>>>>>>> 
>>>>>>>> 
>> <ovirt-vm:volumeID>a2314816-7970-49ce-a80c-ab0d1cf17c78</ovirt-vm:volumeID>
>>>>>>>> 
>>>>>>>> 
>> <ovirt-vm:imageID>a1d56b14-6d72-4f46-a0aa-eb0870c36bc4</ovirt-vm:imageID>
>>>>>>>> 
>>>>>>>> 
>> <ovirt-vm:domainID>781717e5-1cff-43a1-b586-9941503544e8</ovirt-vm:domainID>
>>>>>>>>      </ovirt-vm:device>
>>>>>>>>      <ovirt-vm:launchPaused>false</ovirt-vm:launchPaused>
>>>>>>>>      <ovirt-vm:resumeBehavior>kill</ovirt-vm:resumeBehavior>
>>>>>>>>    </ovirt-vm:vm>
>>>>>>>>  </metadata>
>>>>>>>> </domain>
>>>>>>>> 
>>>>>>>> 2020-02-06 16:38:25,455Z INFO
>>>>>>>> 
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
>>>>>>>> CreateBrokerVDSCommand, return: , log id: 1bfa03c4
>>>>>>>> 2020-02-06 16:38:25,494Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
>> CreateVDSCommand,
>>>>>>>> return: WaitForLaunch, log id: 5e07ba66
>>>>>>>> 2020-02-06 16:38:25,495Z INFO
>>>>>>>> [org.ovirt.engine.core.bll.RunVmCommand]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] Lock freed to
>> object
>>>>>>>> 
>> 'EngineLock:{exclusiveLocks='[df9dbac4-35c0-40ee-acd4-a1cfc959aa8b=VM]',
>>>>>>>> sharedLocks=''}'
>>>>>>>> 2020-02-06 16:38:25,533Z INFO
>>>>>>>> 
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-216) [] EVENT_ID:
>>>>>>>> USER_STARTED_VM(153), VM yumcache was started by
>> admin@internal-authz
>>>>>>>> (Host: node1.ovirt.trashnet.xyz <http://node1.ovirt.trashnet.xyz/>).
>>>>>>>> 2020-02-06 16:38:33,300Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
>>>>>>>> (ForkJoinPool-1-worker-5) [] VM
>> 'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b' was
>>>>>>>> reported as Down on VDS 'c3465ca2-395e-4c0c-b72e-b5b7153df452'(
>>>>>>>> node1.ovirt.trashnet.xyz <http://node1.ovirt.trashnet.xyz/>)
>>>>>>>> 2020-02-06 16:38:33,301Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand]
>>>>>>>> (ForkJoinPool-1-worker-5) [] START, DestroyVDSCommand(HostName =
>>>>>>>> node1.ovirt.trashnet.xyz <http://node1.ovirt.trashnet.xyz/>,
>>>>>>>> 
>> DestroyVmVDSCommandParameters:{hostId='c3465ca2-395e-4c0c-b72e-b5b7153df452',
>>>>>>>> vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b', secondsToWait='0',
>>>>>>>> gracefully='false', reason='', ignoreNoVm='true'}), log id:
>> 1f951ea9
>>>>>>>> 2020-02-06 16:38:33,478Z INFO
>>>>>>>> 
>> [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher]
>>>>>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Fetched 2
>> VMs from
>>>>>>>> VDS 'c3465ca2-395e-4c0c-b72e-b5b7153df452'
>>>>>>>> 2020-02-06 16:38:33,545Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand]
>>>>>>>> (ForkJoinPool-1-worker-5) [] FINISH, DestroyVDSCommand, return:
>> , log id:
>>>>>>>> 1f951ea9
>>>>>>>> 2020-02-06 16:38:33,546Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
>>>>>>>> (ForkJoinPool-1-worker-5) [] VM
>>>>>>>> 'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'(yumcache) moved from
>> 'WaitForLaunch'
>>>>>>>> --> 'Down'
>>>>>>>> 2020-02-06 16:38:33,623Z ERROR
>>>>>>>> 
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>> (ForkJoinPool-1-worker-5) [] EVENT_ID: VM_DOWN_ERROR(119), VM
>> yumcache is
>>>>>>>> down with error. Exit message: internal error: qemu unexpectedly
>> closed the
>>>>>>>> monitor: [2020-02-06 16:38:31.723977] E [MSGID: 108006]
>>>>>>>> [afr-common.c:5323:__afr_handle_child_down_event]
>> 0-vmstore-replicate-0:
>>>>>>>> All subvolumes are down. Going offline until at least one of
>> them comes
>>>>>>>> back up.
>>>>>>>> [2020-02-06 16:38:31.724765] I [io-stats.c:4027:fini] 0-vmstore:
>>>>>>>> io-stats translator unloaded
>>>>>>>> 2020-02-06T16:38:32.573511Z qemu-kvm: -drive file=gluster://
>>>>>>>> 
>> node1.fs.trashnet.xyz:24007/vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78,file.debug=4,format=qcow2,if=none,id=drive-ua
>>  
>> <http://node1.fs.trashnet.xyz:24007/vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78,file.debug=4,format=qcow2,if=none,id=drive-ua>-a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,serial=a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,werror=stop,rerror=stop,cache=none,discard=unmap,aio=native:
>>>>>>>> Could not read qcow2 header: Invalid argument.
>>>>>>>> 2020-02-06 16:38:33,624Z INFO
>>>>>>>> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
>>>>>>>> (ForkJoinPool-1-worker-5) [] add VM
>>>>>>>> 'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'(yumcache) to rerun
>> treatment
>>>>>>>> 2020-02-06 16:38:33,796Z ERROR
>>>>>>>> [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring]
>>>>>>>> (ForkJoinPool-1-worker-5) [] Rerun VM
>>>>>>>> 'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'. Called from VDS '
>>>>>>>> node1.ovirt.trashnet.xyz <http://node1.ovirt.trashnet.xyz/>'
>>>>>>>> 2020-02-06 16:38:33,899Z WARN
>>>>>>>> 
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-223) [] EVENT_ID:
>>>>>>>> USER_INITIATED_RUN_VM_FAILED(151), Failed to run VM yumcache on
>> Host
>>>>>>>> node1.ovirt.trashnet.xyz <http://node1.ovirt.trashnet.xyz/>.
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>
>>>>>>>> To unsubscribe send an email to users-le...@ovirt.org 
>>>>>>>> <mailto:users-le...@ovirt.org>
>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ 
>>>>>>>> <https://www.ovirt.org/site/privacy-policy/>
>>>>>>>> oVirt Code of Conduct:
>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ 
>>>>>>>> <https://www.ovirt.org/community/about/community-guidelines/>
>>>>>>>> List Archives:
>>>>>>>> 
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/6GTBANZ4R44HJE2BU55GAEBLTETUXTKT/
>>  
>> <https://lists.ovirt.org/archives/list/users@ovirt.org/message/6GTBANZ4R44HJE2BU55GAEBLTETUXTKT/>
>>>>>>>> 
>>>>>>> _______________________________________________
>>>>>> Users mailing list -- users@ovirt.org
>>>>>> To unsubscribe send an email to users-le...@ovirt.org
>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>>>> oVirt Code of Conduct:
>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>> List Archives:
>>>>>> 
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/5EZYM4OADD65RVNDIQVS25EGZOK65PWX/
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>> Users mailing list -- users@ovirt.org
>>>>> To unsubscribe send an email to users-le...@ovirt.org
>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>>> oVirt Code of Conduct:
>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>> 
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/KKUVN3ZR5QOVKKCOR5XIUYYC6EL7FYXS/
>>>>> 
>>>> 
>>>> Ce message et toutes les pièces jointes (ci-après le “message”) sont
>>>> établis à l’intention exclusive de ses destinataires et sont
>> confidentiels.
>>>> Si vous recevez ce message par erreur, merci de le détruire et d’en
>> avertir
>>>> immédiatement l’expéditeur. Toute utilisation de ce message non
>> conforme a
>>>> sa destination, toute diffusion ou toute publication, totale ou
>> partielle,
>>>> est interdite, sauf autorisation expresse. L’internet ne permettant
>> pas
>>>> d’assurer l’intégrité de ce message . Interactiv-group (et ses
>> filiales)
>>>> décline(nt) toute responsabilité au titre de ce message, dans
>> l’hypothèse
>>>> ou il aurait été modifié. IT, ES, UK.
>>>> <https://interactiv-group.com/disclaimer.html 
>>>> <https://interactiv-group.com/disclaimer.html>>
>>>> _______________________________________________
>>>> Users mailing list -- users@ovirt.org
>>>> To unsubscribe send an email to users-le...@ovirt.org
>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>> oVirt Code of Conduct:
>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>> List Archives:
>>>> 
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/S2RYKJLRE533BZB5A3ZDDMIZ573W7HTC/
>>>> 
>>> _______________________________________________
>>> Users mailing list -- users@ovirt.org
>>> To unsubscribe send an email to users-le...@ovirt.org
>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>> oVirt Code of Conduct:
>>> https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives:
>>> 
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/23SAGPLPE5MHCGGQJXUK6D7MMO6CFWWJ/
>>> 
>>> 
>>> 
> 
> My guess  is different - a real Load Balancer , like HAproxy, can take  care  
> when a backend server is down.
> 
> Still,  it will be nice to learn Darrell's solution.
> 
> Best Regards,
> Strahil Nikolov

_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZBVBCIAOVIVP46MBQTGJCBC37O4CVF5F/

Reply via email to