[ovirt-users] Re: Slow VM replication

[David Johnson]

It's amazing what replacing the failed drive for the ZIL does for the write
throughput. I peaked at 6 Gbit/second.

Next bottleneck.

*David Johnson*
*Director of Development, Maxis Technology*
844.696.2947 ext 702 (o) | 479.531.3590 (c)




*Follow us:*  


On Fri, Apr 30, 2021 at 12:20 PM Vinícius Ferrão 
wrote:

> As far as I know FreeNAS supports VMware-like snapshots but not the
> language that oVirt speaks.
>
> Another point to observe is that FreeNAS with RAID-Z3 is not recommended
> from VM storage, because it is just slow for this purpose. Usually NFS
> issues sync requests which will be slow due to it's nature. iSCSI tends to
> be faster since it can request sync and async operations.
>
> If you pool does not have a ZIL for Sync Write Offload you'll get this
> kind of performance.
>
> I have three oVirt installs backed by FreeNAS and all of them have 24
> spindles in stripe of mirrors configuration, due to those issues.
>
> Regards,
>
>
> On 30 Apr 2021, at 14:05, Strahil Nikolov via Users 
> wrote:
>
> I have the feeling that it's using the wrong Network.
>
> Also, check if TrueNAS supports NFS Server-Side Copy.
> In order to use that feature mount the share  via v4.2 .
>
>
> Best Regards,
> Strahil Nikolov
>
> On Fri, Apr 30, 2021 at 19:10, David Johnson
>  wrote:
> Hi everyone,
>
> When cloning a VM, I discovered that the time to close appears excessive
> based on the underlying platform I am using. It appears that the clone
> operation is not making efficient use of the network.  I have seen up to 4
> GBits sustained throughput by applications on VM's on the cluster.
>
> Is there a configuration I might be missing?
>
> *System specifics:*
> Backing store: NFS on TrueNAS running zRaid 3 on 11 spinning disks
>
> Ovirt Controller: I7 desktop
> General network: 1 GBit Ethernet
>
> Ovirt Host:  Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz, 2x32 cores, 256
> GB RAM
> Data Network: 10 GBit 10GBase-Twinax, dedicated to this Host and the
> TrueNAS
>
>
> *Operation:*
> Copying the 60GB partition, the copy operation never exceeds 40 megabytes
> per second (less than 0.5 GBit), even though the dedicated 10 Gigabit data
> network is not otherwise busy.
>
> 
>
> *David Johnson*
> *Director of Development, Maxis Technology*
> 844.696.2947 ext 702 (o) | 479.531.3590 (c)
> 
> 
> 
>
> *Follow us:*  
> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WTEWHJ6NMFGGKQHQNGGI3C5TPC2OETGU/
>
> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGJLJUBHAUXA23NHVRXYHY2NL6IYWWVL/
>
>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/7GIHDPHEOJ2THEOG6DZHJA2GYMN4G3CV/

[ovirt-users] Re: Slow VM replication

[David Johnson]

It looks like my SSD for ZIL has failed. That would explain a lot.  I will
also look at the other questions you raised.

Thank you!

*David Johnson*
*Director of Development, Maxis Technology*
844.696.2947 ext 702 (o) | 479.531.3590 (c)




*Follow us:*  


On Fri, Apr 30, 2021 at 12:20 PM Vinícius Ferrão 
wrote:

> As far as I know FreeNAS supports VMware-like snapshots but not the
> language that oVirt speaks.
>
> Another point to observe is that FreeNAS with RAID-Z3 is not recommended
> from VM storage, because it is just slow for this purpose. Usually NFS
> issues sync requests which will be slow due to it's nature. iSCSI tends to
> be faster since it can request sync and async operations.
>
> If you pool does not have a ZIL for Sync Write Offload you'll get this
> kind of performance.
>
> I have three oVirt installs backed by FreeNAS and all of them have 24
> spindles in stripe of mirrors configuration, due to those issues.
>
> Regards,
>
>
> On 30 Apr 2021, at 14:05, Strahil Nikolov via Users 
> wrote:
>
> I have the feeling that it's using the wrong Network.
>
> Also, check if TrueNAS supports NFS Server-Side Copy.
> In order to use that feature mount the share  via v4.2 .
>
>
> Best Regards,
> Strahil Nikolov
>
> On Fri, Apr 30, 2021 at 19:10, David Johnson
>  wrote:
> Hi everyone,
>
> When cloning a VM, I discovered that the time to close appears excessive
> based on the underlying platform I am using. It appears that the clone
> operation is not making efficient use of the network.  I have seen up to 4
> GBits sustained throughput by applications on VM's on the cluster.
>
> Is there a configuration I might be missing?
>
> *System specifics:*
> Backing store: NFS on TrueNAS running zRaid 3 on 11 spinning disks
>
> Ovirt Controller: I7 desktop
> General network: 1 GBit Ethernet
>
> Ovirt Host:  Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz, 2x32 cores, 256
> GB RAM
> Data Network: 10 GBit 10GBase-Twinax, dedicated to this Host and the
> TrueNAS
>
>
> *Operation:*
> Copying the 60GB partition, the copy operation never exceeds 40 megabytes
> per second (less than 0.5 GBit), even though the dedicated 10 Gigabit data
> network is not otherwise busy.
>
> 
>
> *David Johnson*
> *Director of Development, Maxis Technology*
> 844.696.2947 ext 702 (o) | 479.531.3590 (c)
> 
> 
> 
>
> *Follow us:*  
> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WTEWHJ6NMFGGKQHQNGGI3C5TPC2OETGU/
>
> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGJLJUBHAUXA23NHVRXYHY2NL6IYWWVL/
>
>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DLYFFEYXNTUA6344OLIB36N7C7SJSEMC/

[ovirt-users] Re: Slow VM replication

[Vinícius Ferrão via Users]

As far as I know FreeNAS supports VMware-like snapshots but not the language 
that oVirt speaks.

Another point to observe is that FreeNAS with RAID-Z3 is not recommended from 
VM storage, because it is just slow for this purpose. Usually NFS issues sync 
requests which will be slow due to it's nature. iSCSI tends to be faster since 
it can request sync and async operations.

If you pool does not have a ZIL for Sync Write Offload you'll get this kind of 
performance.

I have three oVirt installs backed by FreeNAS and all of them have 24 spindles 
in stripe of mirrors configuration, due to those issues.

Regards,


On 30 Apr 2021, at 14:05, Strahil Nikolov via Users 
mailto:users@ovirt.org>> wrote:

I have the feeling that it's using the wrong Network.

Also, check if TrueNAS supports NFS Server-Side Copy.
In order to use that feature mount the share  via v4.2 .


Best Regards,
Strahil Nikolov

On Fri, Apr 30, 2021 at 19:10, David Johnson
mailto:djohn...@maxistechnology.com>> wrote:
Hi everyone,

When cloning a VM, I discovered that the time to close appears excessive based 
on the underlying platform I am using. It appears that the clone operation is 
not making efficient use of the network.  I have seen up to 4 GBits sustained 
throughput by applications on VM's on the cluster.

Is there a configuration I might be missing?

System specifics:
Backing store: NFS on TrueNAS running zRaid 3 on 11 spinning disks

Ovirt Controller: I7 desktop
General network: 1 GBit Ethernet

Ovirt Host:  Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz, 2x32 cores, 256 GB RAM
Data Network: 10 GBit 10GBase-Twinax, dedicated to this Host and the TrueNAS


Operation:
Copying the 60GB partition, the copy operation never exceeds 40 megabytes per 
second (less than 0.5 GBit), even though the dedicated 10 Gigabit data network 
is not otherwise busy.



David Johnson
Director of Development, Maxis Technology
844.696.2947 ext 702 (o) | 479.531.3590 (c)
[https://maxistechnology.com/wp-content/uploads/2021/04/linkedin.png]
 [https://maxistechnology.com/wp-content/uploads/2021/04/email.png] 


[https://maxistechnology.com/wp-content/uploads/2021/04/Maxis_logo_small.gif]

Follow us: 
[https://maxistechnology.com/wp-content/uploads/2021/04/linkedin.png] 


___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to 
users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/WTEWHJ6NMFGGKQHQNGGI3C5TPC2OETGU/
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to 
users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGJLJUBHAUXA23NHVRXYHY2NL6IYWWVL/

___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/46E3UOOTVXJSO2BUNQ4AIJR3XQJ36SGM/

[ovirt-users] Unable to migrate Engine to another HE Host

[Marko Vrgotic]

Dear oVirt,

I have already reached out twice regarding the issues that occurred, due to 
power outage, but noticed only when upgrading engine to latest 4.3. version.

I am unable to redeploy engine on Host2, the hosted-engine file stays empty and 
VDSM on Hosts1 and 3 is reporting, even though I cleared the metadata for the 
Host2, on Host 1 and Host3:

2021-04-30 05:57:58,454-0700 ERROR (jsonrpc/7) 
[ovirt_hosted_engine_ha.client.client.HAClient] Malformed metadata for host 2: 
received 0 of 512 expected bytes (client:137)

Today I tried to migrate HE from Host 3 to Host 1 and it fails each time with 
following message:

On Engine:
2021-04-30 12:57:56,961Z ERROR 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(EE-ManagedThreadFactory-engine-Thread-1233892) [] EVENT_ID: 
VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed  (VM: HostedEngine, 
Source: ovirt-sj-03.ictv.com, Destination: ovirt-sj-01.ictv.com).

On source Host:
2021-04-30 05:57:56,705-0700 ERROR (migsrc/66b6d489) [virt.vm] 
(vmId='66b6d489-ceb8-486a-951a-355e21f13627') Failed to migrate (migration:450)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 431, in 
_regular_run
time.time(), migrationParams, machineParams
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 505, in 
_startUnderlyingMigration
self._perform_with_conv_schedule(duri, muri)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 591, in 
_perform_with_conv_schedule
self._perform_migration(duri, muri)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 525, in 
_perform_migration
self._migration_flags)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 100, in f
ret = attr(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", 
line 131, in wrapper
ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in 
wrapper
return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1781, in 
migrateToURI3
if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', 
dom=self)
libvirtError: operation aborted: migration out job: canceled by client


I know that this version is end of life – but I would very much appreciate if 
someone could help me asses if this means corruption in DB or the overall 
damage, simply to know how to plan further actions.
My impression was that I still had to functional HE Hosts in the pool, but 
after seeing migration failure, it’s pretty much down to single host.

This is production system, so I cannot just move on to upgrading/deploying to 
4.4.

Additionally – :

  *   Is the effect of the engine-cleanup on HE Host local or it affects all HE 
Hosts? Could that help bringing the Host back to state so that HE can be 
re-deployed?
  *   What is the effect or reinitialize-lockspace?

Kindly awaiting your reply. Happy to provide any additional information needed.



-
kind regards/met vriendelijke groeten

Marko Vrgotic
Sr. System Engineer @ System Administration

ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgo...@activevideo.com
w: www.activevideo.com

ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ 
Hilversum, The Netherlands. The information contained in this message may be 
legally privileged and confidential. It is intended to be read only by the 
individual or entity to whom it is addressed or by their designee. If the 
reader of this message is not the intended recipient, you are on notice that 
any distribution of this message, in any form, is strictly prohibited.  If you 
have received this message in error, please immediately notify the sender 
and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or 
destroy any copy of this message.


___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FRTRNTSGPLQOS72JDPSE766WWROFGTBA/

[ovirt-users] Re: oVirt 4.3 DWH with Grafana

[Nardus Geldenhuys]

Hi

Thank you. I did figured it out after a while...

Another question. How can I see LUN utilization ? We had an issue where one LUN 
was over utiliazed, meaning it was hammered by the VM's. We only discovered 
this after the SAN team told us, we moved the VM's storage around it we 
resolved the issue. Is it possible to get a Grafana graph with LUN utilization 
per VM?

Thanks

Nardus
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/NJXX3UBTJE54L7XWKRMS27SOHFW6K4Q2/

[ovirt-users] error id 980

[ozmen62]

Hi,
First of all, thanks for this great list and the people who try to help 
eachother.

This time my problem is about engine OVF file or engine storage setting.

How can i came with this idea, because every 2-3 hours engine try to migrate 
other host(From host1 to host2)

"Invalid status on Data Center B300. Setting status to Non Responsive."

when i check hosts log , see these

ovirt-ha-agent 
ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed 
extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf
ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR 
Engine VM stopped on localhost
ovirt-ha-agent ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore ERROR Unable 
to extract HEVM OVF

i believe there is some kind of fc storage path or config problem. My storage 
setting is good. there is no error about them

Is there anyone who experienced about that to solve it

thanks
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/GL6CNZJPIXWZBNPY3PPMLDOH6V23MFOI/

[ovirt-users] Re: new engine restore from backup

[ozmen62]

4.4.5-11

I found the problem. Backup tool did not take the engine.db
So I had to clean engine install and import all FC disks.
After that one by one i processed virtual machines. It was the only way.
Now , i wrote additional bash script to take pgsql full dump
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/W77JVZV47S3TR4RZQRRLYZDGIDY6X3WP/

[ovirt-users] Re: [OLVM] Host non responsive after installation

[Ales Musil]

On Wed, Apr 28, 2021 at 3:09 PM  wrote:

> oVirt version: 4.3
> - ovirt-engine-appliance.x86_64 4.3-20200603.1.0.2.el7
> vdsm version: 4.30.46-1.0.3.el7
>
> I can connect to the host manually. If I supply invalid SSH args when
> adding the host, it fails immediately rather than running for a while and
> failing later.
>

Can you please share vdsm.log and supervdsm.log from the host?

Thanks,
Ales



> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/3IZYCB37KK2VXTIZHEHJQ6WQOWAEZHAC/
>


-- 

Ales Musil

Software Engineer - RHV Network

Red Hat EMEA 

amu...@redhat.comIM: amusil

___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/TVINIDHOD63TG77O4K5O7VQRSMUU5423/

[ovirt-users] Migration from deprecated OpenStack provider to cinderlib

[Sandro Bonazzola]

Il giorno ven 30 apr 2021 alle ore 08:48 Konstantin Shalygin 
ha scritto:

> Hi Sandro,
>
> The question is - will ovirt plan to provide database migration scripts
> from deprecated OpenStack provider to cinderlib? I mean put in survey
> actual users and quantity of images in domain
>

Moving this question to its own thread.
+Eyal Shenitzky  can you please reply


>
>
> Thanks,
> k
>
> Sent from my iPhone
>
>

-- 

Sandro Bonazzola

MANAGER, SOFTWARE ENGINEERING, EMEA R RHV

Red Hat EMEA 

sbona...@redhat.com


*Red Hat respects your work life balance. Therefore there is no need to
answer this email out of your office hours.
*
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IF3KQN2RKRMFVF4HGPI2WWJBEQXSXYRY/

[ovirt-users] Re: oVirt 2021 Spring survey questions

[Konstantin Shalygin]

Hi Sandro,

The question is - will ovirt plan to provide database migration scripts from 
deprecated OpenStack provider to cinderlib? I mean put in survey actual users 
and quantity of images in domain


Thanks,
k

Sent from my iPhone
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/JCNNQPHTJDW6NAO22A5VNW7LAHHT6ZKJ/