[ovirt-users] Performance issue to retrieve VM's after upgrading engine from 4.2 to 4.3

2021-01-25 Thread Jonas Lindholm 
This is a weird issue.
Before upgrading it took 8-9 seconds to retrieve all VM's, 1230, using 
/ovirt-engine/api/vms URI. After the upgrade it now takes over 2 minutes.
Query other items such as hosts, clusters, data centers etc is fast and the 
same speed as prior the upgrade from 4.2 to 4.3
I don't know if this is DB related or jboss related but any clue where to start 
to dig deeper is welcome.
I also built a new VM and restored the DB's and ran engine-setup, but that new 
engine also gets more or less exactly the same response times for the VM's 
using the restAPI.

I have another engine with same version 4.3.10.4-1, That only have 261 VMs but 
those are retrieved within 1-2 seconds only so I don't understand how that 
engine is so much faster than the one I upgraded.
The UI also has the same long response times to show VM's from Compute -> 
Virtual Machines

Any idea what could cause this?
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IXSGJMOFP5MWLLBGRY4XFD5WFJGVNK64/

Re: [ovirt-users] Performance Issue

2018-04-18 Thread Tony Brian Albers 
On 18/04/18 14:54, Gianluca Cecchi wrote:
> On Wed, Apr 18, 2018 at 2:25 PM, Tony Brian Albers  > wrote:
> 
> You need a lot faster disks to keep up with all that random I/O.
> 
> "Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD
>   >     Raid )"
> 
> Ehm, are you running VM's on a DD box?  Why on earth would you do that?
> 
> /tony
> 
> 
> On 2018-04-18 14:09, Thomas Fecke wrote:
> > Atleast we found the Bottle neck. Our Data Domain is the Problem
> > 
> 
> 
> I guess and hope he made a mix between Data Store and Storage domain ;-)
> 

Hopefully, but I've seen people running vmware on isilon. Guess how that 
worked out for them.. ;)

/tony

-- 
Tony Albers
Systems administrator, IT-development
Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark.
Tel: +45 2566 2383 / +45 8946 2316
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-18 Thread Gianluca Cecchi 
On Wed, Apr 18, 2018 at 2:25 PM, Tony Brian Albers  wrote:

> You need a lot faster disks to keep up with all that random I/O.
>
> "Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD
>  > Raid )"
>
> Ehm, are you running VM's on a DD box?  Why on earth would you do that?
>
> /tony
>
>
> On 2018-04-18 14:09, Thomas Fecke wrote:
> > Atleast we found the Bottle neck. Our Data Domain is the Problem
> >


I guess and hope he made a mix between Data Store and Storage domain ;-)
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-18 Thread Tony Brian Albers 
You need a lot faster disks to keep up with all that random I/O.

"Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD
 > Raid )"

Ehm, are you running VM's on a DD box?  Why on earth would you do that?

/tony


On 2018-04-18 14:09, Thomas Fecke wrote:
> Atleast we found the Bottle neck. Our Data Domain is the Problem
> 
> IOStat:
> 
> Device: rrqm/s   wrqm/s r/s w/s    rkB/s    wkB/s 
> avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
> 
> sda   0,00   615,67   78,00  962,33  2125,33 37065,33
> 75,34 5,68    5,42    1,10    5,77   0,96  99,40
> 
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
> 
>     0,00    0,00    1,44   22,63    0,00   75,93
> 
> Device: rrqm/s   wrqm/s r/s w/s    rkB/s    wkB/s 
> avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
> 
> sda   0,67   655,33   92,00 1033,67  2446,67 38390,67
> 72,56 5,66    5,06    1,83    5,35   0,88  99,50
> 
> Any Idea why we got so much write Requests? Its getting up to 2000.
> 
> I guess the IO is from our Windows VM´s
> 
> *From:*users-boun...@ovirt.org  *On Behalf Of 
> *Thomas Fecke
> *Sent:* Dienstag, 17. April 2018 16:18
> *To:* Roy Golan 
> *Cc:* users@ovirt.org
> *Subject:* Re: [ovirt-users] Performance Issue
> 
> Guest: Win 10 and 2016
> 
> Disk: VirtISCSI – Thin provision
> 
> Domain: Single Data Domain – NFS 4
> 
> *From:*Roy Golan 
> *Sent:* Dienstag, 17. April 2018 12:58
> *To:* Thomas Fecke 
> *Cc:* users@ovirt.org
> *Subject:* Re: [ovirt-users] Performance Issue
> 
> On Tue, 17 Apr 2018 at 12:58 Thomas Fecke  <mailto:thomas.fe...@eset.de>> wrote:
> 
> Hey Thank you,
> 
> If been monitoring vor about an Hour now. The Templates VM´s are
> really slow – copy a new Template is slow. Non Template VM´s are fast
> 
> Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD
> Raid )
> 
> Network is on 500 Mbits/s internal ( 10 Gbit Stroage connection )
> 
> And about 25Mbit/s external ( 400.000 k Internet )
> 
> I realy cant find that bottle neck
> 
> 
> what type is the disks, the domain, how many domains ?
> 
> *From:*Roy Golan mailto:rgo...@redhat.com>>
> *Sent:* Dienstag, 17. April 2018 11:52
> *To:* Thomas Fecke mailto:thomas.fe...@eset.de>>
> *Cc:* users@ovirt.org <mailto:users@ovirt.org>
> *Subject:* Re: [ovirt-users] Performance Issue
> 
> On Tue, 17 Apr 2018 at 12:45 Thomas Fecke  <mailto:thomas.fe...@eset.de>> wrote:
> 
> Okay,
> 
> it seems to be that the Storage IO is the Problem – every Write
> and Read Process takes a lot of time
> 
> Sometimes the copy Job stops with 0/Byte and the traffic goes up
> and down like a mountain.
> 
> But the Storage Read and Write M/s looking fine… I don’t get it
> 
> Any toughts?
> 
> Keep monitoring you storage backend interface and see what's going
> on there if you don't see anything special on the host. It could be
> the network that leads to slow IO as well, who knows.
> 
> If the initial VM creation is taking long you might want to create a
> pool from your template with Pre-Started VMs, and that would at
> least save you from the wait when you actually need the VM.
> 
> *From:* users-boun...@ovirt.org <mailto:users-boun...@ovirt.org>
> mailto:users-boun...@ovirt.org>> *On
> Behalf Of *Thomas Fecke
> *Sent:* Dienstag, 17. April 2018 10:58
> *To:* users@ovirt.org <mailto:users@ovirt.org>
> *Subject:* [ovirt-users] Performance Issue
> 
> Hey Guys,
> 
> We Deploy a lot of Templates. We got our Training Environment
> build in Ovirt.
> 
> Our Problem – when we Deploy the Same Template. The VM´s getting
> slower every Time we deply a new VM from that Template.
> 
> And I really don’t know why. RAM locking good – CPU and Network
> looking good.
> 
> But it take about 15 Minute to create a new Template based VM –
> normaly it takes about 30 Seconds.
> 
> I checked nload and the Interface traffic – but its not really high.
> 
> Can someone explain why its getting so slow and how to troubleshoot?
> 
> ___
> Users mailing list
> Users@ovirt.org <mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users
> 
> 
> 
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> 


-- 
Tony Albers
Systems administrator, IT-development
Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark.
Tel: +45 2566 2383 / +45 8946 2316
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-18 Thread Thomas Fecke 
Atleast we found the Bottle neck. Our Data Domain is the Problem

IOStat:

Device: rrqm/s   wrqm/s r/s w/srkB/swkB/s avgrq-sz 
avgqu-sz   await r_await w_await  svctm  %util
sda   0,00   615,67   78,00  962,33  2125,33 37065,3375,34 
5,685,421,105,77   0,96  99,40

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
   0,000,001,44   22,630,00   75,93

Device: rrqm/s   wrqm/s r/s w/srkB/swkB/s avgrq-sz 
avgqu-sz   await r_await w_await  svctm  %util
sda   0,67   655,33   92,00 1033,67  2446,67 38390,6772,56 
5,665,061,835,35   0,88  99,50


Any Idea why we got so much write Requests? Its getting up to 2000.

I guess the IO is from our Windows VM´s







From: users-boun...@ovirt.org  On Behalf Of Thomas 
Fecke
Sent: Dienstag, 17. April 2018 16:18
To: Roy Golan 
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Performance Issue

Guest: Win 10 and 2016
Disk: VirtISCSI – Thin provision
Domain: Single Data Domain – NFS 4


From: Roy Golan 
Sent: Dienstag, 17. April 2018 12:58
To: Thomas Fecke 
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Performance Issue


On Tue, 17 Apr 2018 at 12:58 Thomas Fecke 
mailto:thomas.fe...@eset.de>> wrote:
Hey Thank you,

If been monitoring vor about an Hour now. The Templates VM´s are really slow – 
copy a new Template is slow. Non Template VM´s are fast

Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD Raid )
Network is on 500 Mbits/s internal ( 10 Gbit Stroage connection )
And about 25Mbit/s external ( 400.000 k Internet )

I realy cant find that bottle neck


what type is the disks, the domain, how many domains ?

From: Roy Golan mailto:rgo...@redhat.com>>
Sent: Dienstag, 17. April 2018 11:52
To: Thomas Fecke mailto:thomas.fe...@eset.de>>
Cc: users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] Performance Issue


On Tue, 17 Apr 2018 at 12:45 Thomas Fecke 
mailto:thomas.fe...@eset.de>> wrote:
Okay,

it seems to be that the Storage IO is the Problem – every Write and Read 
Process takes a lot of time

Sometimes the copy Job stops with 0/Byte and the traffic goes up and down like 
a mountain.


But the Storage Read and Write M/s looking fine… I don’t get it

Any toughts?


Keep monitoring you storage backend interface and see what's going on there if 
you don't see anything special on the host. It could be the network that leads 
to slow IO as well, who knows.
If the initial VM creation is taking long you might want to create a pool from 
your template with Pre-Started VMs, and that would at least save you from the 
wait when you actually need the VM.

From: users-boun...@ovirt.org<mailto:users-boun...@ovirt.org> 
mailto:users-boun...@ovirt.org>> On Behalf Of Thomas 
Fecke
Sent: Dienstag, 17. April 2018 10:58
To: users@ovirt.org<mailto:users@ovirt.org>
Subject: [ovirt-users] Performance Issue

Hey Guys,

We Deploy a lot of Templates. We got our Training Environment build in Ovirt.

Our Problem – when we Deploy the Same Template. The VM´s getting slower every 
Time we deply a new VM from that Template.

And I really don’t know why. RAM locking good – CPU and Network looking good.

But it take about 15 Minute to create a new Template based VM – normaly it 
takes about 30 Seconds.

I checked nload and the Interface traffic – but its not really high.

Can someone explain why its getting so slow and how to troubleshoot?

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-17 Thread Thomas Fecke 
Guest: Win 10 and 2016
Disk: VirtISCSI – Thin provision
Domain: Single Data Domain – NFS 4



From: Roy Golan 
Sent: Dienstag, 17. April 2018 12:58
To: Thomas Fecke 
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Performance Issue


On Tue, 17 Apr 2018 at 12:58 Thomas Fecke 
mailto:thomas.fe...@eset.de>> wrote:
Hey Thank you,

If been monitoring vor about an Hour now. The Templates VM´s are really slow – 
copy a new Template is slow. Non Template VM´s are fast

Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD Raid )
Network is on 500 Mbits/s internal ( 10 Gbit Stroage connection )
And about 25Mbit/s external ( 400.000 k Internet )

I realy cant find that bottle neck


what type is the disks, the domain, how many domains ?

From: Roy Golan mailto:rgo...@redhat.com>>
Sent: Dienstag, 17. April 2018 11:52
To: Thomas Fecke mailto:thomas.fe...@eset.de>>
Cc: users@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] Performance Issue


On Tue, 17 Apr 2018 at 12:45 Thomas Fecke 
mailto:thomas.fe...@eset.de>> wrote:
Okay,

it seems to be that the Storage IO is the Problem – every Write and Read 
Process takes a lot of time

Sometimes the copy Job stops with 0/Byte and the traffic goes up and down like 
a mountain.


But the Storage Read and Write M/s looking fine… I don’t get it

Any toughts?


Keep monitoring you storage backend interface and see what's going on there if 
you don't see anything special on the host. It could be the network that leads 
to slow IO as well, who knows.
If the initial VM creation is taking long you might want to create a pool from 
your template with Pre-Started VMs, and that would at least save you from the 
wait when you actually need the VM.

From: users-boun...@ovirt.org<mailto:users-boun...@ovirt.org> 
mailto:users-boun...@ovirt.org>> On Behalf Of Thomas 
Fecke
Sent: Dienstag, 17. April 2018 10:58
To: users@ovirt.org<mailto:users@ovirt.org>
Subject: [ovirt-users] Performance Issue

Hey Guys,

We Deploy a lot of Templates. We got our Training Environment build in Ovirt.

Our Problem – when we Deploy the Same Template. The VM´s getting slower every 
Time we deply a new VM from that Template.

And I really don’t know why. RAM locking good – CPU and Network looking good.

But it take about 15 Minute to create a new Template based VM – normaly it 
takes about 30 Seconds.

I checked nload and the Interface traffic – but its not really high.

Can someone explain why its getting so slow and how to troubleshoot?

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-17 Thread Roy Golan 
On Tue, 17 Apr 2018 at 12:58 Thomas Fecke  wrote:

> Hey Thank you,
>
>
>
> If been monitoring vor about an Hour now. The Templates VM´s are really
> slow – copy a new Template is slow. Non Template VM´s are fast
>
>
>
> Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD Raid )
>
> Network is on 500 Mbits/s internal ( 10 Gbit Stroage connection )
>
> And about 25Mbit/s external ( 400.000 k Internet )
>
>
>
> I realy cant find that bottle neck
>
>
>

what type is the disks, the domain, how many domains ?


> *From:* Roy Golan 
> *Sent:* Dienstag, 17. April 2018 11:52
> *To:* Thomas Fecke 
> *Cc:* users@ovirt.org
> *Subject:* Re: [ovirt-users] Performance Issue
>
>
>
>
>
> On Tue, 17 Apr 2018 at 12:45 Thomas Fecke  wrote:
>
> Okay,
>
>
>
> it seems to be that the Storage IO is the Problem – every Write and Read
> Process takes a lot of time
>
>
>
> Sometimes the copy Job stops with 0/Byte and the traffic goes up and down
> like a mountain.
>
>
>
>
>
> But the Storage Read and Write M/s looking fine… I don’t get it
>
>
>
> Any toughts?
>
>
>
>
>
> Keep monitoring you storage backend interface and see what's going on
> there if you don't see anything special on the host. It could be the
> network that leads to slow IO as well, who knows.
>
> If the initial VM creation is taking long you might want to create a pool
> from your template with Pre-Started VMs, and that would at least save you
> from the wait when you actually need the VM.
>
>
>
> *From:* users-boun...@ovirt.org  *On Behalf Of 
> *Thomas
> Fecke
> *Sent:* Dienstag, 17. April 2018 10:58
> *To:* users@ovirt.org
> *Subject:* [ovirt-users] Performance Issue
>
>
>
> Hey Guys,
>
>
>
> We Deploy a lot of Templates. We got our Training Environment build in
> Ovirt.
>
>
>
> Our Problem – when we Deploy the Same Template. The VM´s getting slower
> every Time we deply a new VM from that Template.
>
>
>
> And I really don’t know why. RAM locking good – CPU and Network looking
> good.
>
>
>
> But it take about 15 Minute to create a new Template based VM – normaly it
> takes about 30 Seconds.
>
>
>
> I checked nload and the Interface traffic – but its not really high.
>
>
>
> Can someone explain why its getting so slow and how to troubleshoot?
>
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-17 Thread Thomas Fecke 
Hey Thank you,

If been monitoring vor about an Hour now. The Templates VM´s are really slow – 
copy a new Template is slow. Non Template VM´s are fast

Storage IO is average on 75 M/s Read Disk and 25 Write Disk. ( SSD Raid )
Network is on 500 Mbits/s internal ( 10 Gbit Stroage connection )
And about 25Mbit/s external ( 400.000 k Internet )

I realy cant find that bottle neck

From: Roy Golan 
Sent: Dienstag, 17. April 2018 11:52
To: Thomas Fecke 
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Performance Issue


On Tue, 17 Apr 2018 at 12:45 Thomas Fecke 
mailto:thomas.fe...@eset.de>> wrote:
Okay,

it seems to be that the Storage IO is the Problem – every Write and Read 
Process takes a lot of time

Sometimes the copy Job stops with 0/Byte and the traffic goes up and down like 
a mountain.


But the Storage Read and Write M/s looking fine… I don’t get it

Any toughts?


Keep monitoring you storage backend interface and see what's going on there if 
you don't see anything special on the host. It could be the network that leads 
to slow IO as well, who knows.
If the initial VM creation is taking long you might want to create a pool from 
your template with Pre-Started VMs, and that would at least save you from the 
wait when you actually need the VM.

From: users-boun...@ovirt.org<mailto:users-boun...@ovirt.org> 
mailto:users-boun...@ovirt.org>> On Behalf Of Thomas 
Fecke
Sent: Dienstag, 17. April 2018 10:58
To: users@ovirt.org<mailto:users@ovirt.org>
Subject: [ovirt-users] Performance Issue

Hey Guys,

We Deploy a lot of Templates. We got our Training Environment build in Ovirt.

Our Problem – when we Deploy the Same Template. The VM´s getting slower every 
Time we deply a new VM from that Template.

And I really don’t know why. RAM locking good – CPU and Network looking good.

But it take about 15 Minute to create a new Template based VM – normaly it 
takes about 30 Seconds.

I checked nload and the Interface traffic – but its not really high.

Can someone explain why its getting so slow and how to troubleshoot?

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-17 Thread Roy Golan 
On Tue, 17 Apr 2018 at 12:45 Thomas Fecke  wrote:

> Okay,
>
>
>
> it seems to be that the Storage IO is the Problem – every Write and Read
> Process takes a lot of time
>
>
>
> Sometimes the copy Job stops with 0/Byte and the traffic goes up and down
> like a mountain.
>
>
>
>
>
> But the Storage Read and Write M/s looking fine… I don’t get it
>
>
>
> Any toughts?
>
>
>

Keep monitoring you storage backend interface and see what's going on there
if you don't see anything special on the host. It could be the network that
leads to slow IO as well, who knows.

If the initial VM creation is taking long you might want to create a pool
from your template with Pre-Started VMs, and that would at least save you
from the wait when you actually need the VM.

>
>
> *From:* users-boun...@ovirt.org  *On Behalf Of 
> *Thomas
> Fecke
> *Sent:* Dienstag, 17. April 2018 10:58
> *To:* users@ovirt.org
> *Subject:* [ovirt-users] Performance Issue
>
>
>
> Hey Guys,
>
>
>
> We Deploy a lot of Templates. We got our Training Environment build in
> Ovirt.
>
>
>
> Our Problem – when we Deploy the Same Template. The VM´s getting slower
> every Time we deply a new VM from that Template.
>
>
>
> And I really don’t know why. RAM locking good – CPU and Network looking
> good.
>
>
>
> But it take about 15 Minute to create a new Template based VM – normaly it
> takes about 30 Seconds.
>
>
>
> I checked nload and the Interface traffic – but its not really high.
>
>
>
> Can someone explain why its getting so slow and how to troubleshoot?
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance Issue

2018-04-17 Thread Thomas Fecke 
Okay,

it seems to be that the Storage IO is the Problem - every Write and Read 
Process takes a lot of time

Sometimes the copy Job stops with 0/Byte and the traffic goes up and down like 
a mountain.


But the Storage Read and Write M/s looking fine... I don't get it

Any toughts?


From: users-boun...@ovirt.org  On Behalf Of Thomas 
Fecke
Sent: Dienstag, 17. April 2018 10:58
To: users@ovirt.org
Subject: [ovirt-users] Performance Issue

Hey Guys,

We Deploy a lot of Templates. We got our Training Environment build in Ovirt.

Our Problem - when we Deploy the Same Template. The VM´s getting slower every 
Time we deply a new VM from that Template.

And I really don't know why. RAM locking good - CPU and Network looking good.

But it take about 15 Minute to create a new Template based VM - normaly it 
takes about 30 Seconds.

I checked nload and the Interface traffic - but its not really high.

Can someone explain why its getting so slow and how to troubleshoot?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Performance Issue

2018-04-17 Thread Thomas Fecke 
Hey Guys,

We Deploy a lot of Templates. We got our Training Environment build in Ovirt.

Our Problem - when we Deploy the Same Template. The VM´s getting slower every 
Time we deply a new VM from that Template.

And I really don't know why. RAM locking good - CPU and Network looking good.

But it take about 15 Minute to create a new Template based VM - normaly it 
takes about 30 Seconds.

I checked nload and the Interface traffic - but its not really high.

Can someone explain why its getting so slow and how to troubleshoot?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance issue for GlusterFS as the block storage for VMs

2015-01-15 Thread Donny Davis 
Sure can

 

http://www.gluster.org/community/documentation/index.php/GlusterFS_iSCSI

 

As for tuning, I don't have enough information about my own system to even
say. I know that there are tuning options that you have with glusterFS, and
you will have to see what works best for your deployment. 

 

Regards

Donny D

cloudspin.me

 

From: Yue, Cong [mailto:cong_...@alliedtelesis.com] 
Sent: Thursday, January 15, 2015 12:30 PM
To: Donny Davis; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block
storage for VMs

 

It is just the default configuration. What is your advice for the tuning of
I/O performance?

 

For Gluster FS, actually I used Xenserver currently. I justed mounted the
gluster as a NFS to XenHost to be used as a SR. Is there any way to use
GlusterFS as a iscsi target ?

 

Thanks,

Cong

 

 

From: Donny Davis [mailto:do...@cloudspin.me] 
Sent: Thursday, January 15, 2015 11:24 AM
To: Yue, Cong; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block
storage for VMs

 

I see. 

 

So have you done any tuning for IO performance or are the configs straight
out of the box. You also said you mounted the volume to a vm. Did you mount
it as gluster or use the built in NFS??

 

Donny

 

From: Yue, Cong [mailto:cong_...@alliedtelesis.com] 
Sent: Thursday, January 15, 2015 11:40 AM
To: Donny Davis; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block
storage for VMs

 

I am using iometer (http://www.iometer.org/) to test the IOPS from one of
the vm.

As for IOPS is more trying to test the performance of block, rather than
real file transfer.

 

In my environment, I am using 10Gbe to make two gluster nodes be replicated.
And I mount it to vm as a volume. I tested the performance both for SAS and
SSD.

This is my current result with iometer. 

 


Test ID

Application

Block Size (Bytes)

Read/Write %

Random/Sequential %

I/O Performance Metrics

iscsi-hdd

iscsi-ssd

Guster,HDD

Gluster,SSD


1

Web File Server

4K

95%/5%

75%/25%

IOPS

195.28

2400.63

208.50

646.75


2

Web File Server

8K

95%/5%

75%/25%

IOPS

193.97

2225.73

179.89

649.93


3

Web File Server

64K

95%/5%

75%/25%

IOPS

180.50

1055.05

158.28

402.15


4

Database Online Transaction Processing

8K

70%/30%

100%/0%

IOPS

163.60

2415.90

132.13

308.46


5

Exchange Email

4K

67%/33%

100%/0%

IOPS

167.03

2685.33

145.92

294.56


6

OS Drive

8K

70%/30%

100%/0%

IOPS

163.60

2407.02

146.11

310.82


7

Decision Support System

1M

100%/0%

100%/0%

IOPS

74.24

207.42

81.14

112.20


8

File Server

8K

90%/10%

75%/25%

IOPS

191.20

2102.32

359.54

526.86


9

Video on Demand

512K

100%/0%

100%/0%

IOPS

100.32

327.19

136.66

162.48


10

Traffic Simulation

8K

50%/50%

75%/25%

IOPS

238.28

1923.91

301.05

297.70


11

Web Server Logging

8K

0%/100%

0%/100%

IOPS

3488.47

3644.33

290.33

282.04


12

SQL Server Logging

64K

0%/100%

0%/100%

IOPS

1423.29

1375.33

182.67

168.64


13

OS Paging

64K

90%/10%

0%/100%

IOPS

1215.74

1211.01

381.17

355.99


14

Media Streaming

64K

98%/2%

0%/100%

IOPS

1350.96

1365.22

457.28

455.49

 

The issue for GlusterFS I found is 

-  It can not retain the nature of SSD and SAS, such as SAS is
strong  for sequential access. SSD is strong for random access.

-  In some case, especially for SSD, the performance is pretty bad.

 

Thanks,

Cong

 

 

From: Donny Davis [mailto:do...@cloudspin.me] 
Sent: Thursday, January 15, 2015 10:27 AM
To: Yue, Cong; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block
storage for VMs

 

Do you have any metrics to give an idea of the difference. I am using NFS
right now, and I am migrating to Gluster. I have the gluster system up, and
I see that it seems to provision disks faster than my NFS. I haven't used
any real measurement tools to get actual metrics, this is all perceived. 

 

Do you have an operational gluster?

Do what are you using right now?

 

Donny D

cloudspin.me

 

From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of
Yue, Cong
Sent: Thursday, January 15, 2015 10:57 AM
To: users@ovirt.org
Subject: [ovirt-users] Performance issue for GlusterFS as the block storage
for VMs

 

Hi 

 

I have one question about whether GlusterFS is the suitable solution to be
used as the block storage for VMs.

The failure tolerant and scalability is good for GlusterFS, but in my test,
it seems the iops is pretty bad. In some blog, it said, it is even with
worse performance than normal NFS.

Should I use iscsi+drbd for the block storage for VMs.

Can somebody give some advice for this? 

 

Thanks,
Cong

 

  _  

This e-mail message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you a

Re: [ovirt-users] Performance issue for GlusterFS as the block storage for VMs

2015-01-15 Thread Yue, Cong 
It is just the default configuration. What is your advice for the tuning of I/O 
performance?

For Gluster FS, actually I used Xenserver currently. I justed mounted the 
gluster as a NFS to XenHost to be used as a SR. Is there any way to use 
GlusterFS as a iscsi target ?

Thanks,
Cong


From: Donny Davis [mailto:do...@cloudspin.me]
Sent: Thursday, January 15, 2015 11:24 AM
To: Yue, Cong; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block storage 
for VMs

I see.

So have you done any tuning for IO performance or are the configs straight out 
of the box. You also said you mounted the volume to a vm. Did you mount it as 
gluster or use the built in NFS??

Donny

From: Yue, Cong [mailto:cong_...@alliedtelesis.com]
Sent: Thursday, January 15, 2015 11:40 AM
To: Donny Davis; users@ovirt.org<mailto:users@ovirt.org>
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block storage 
for VMs

I am using iometer (http://www.iometer.org/) to test the IOPS from one of the 
vm.
As for IOPS is more trying to test the performance of block, rather than real 
file transfer.

In my environment, I am using 10Gbe to make two gluster nodes be replicated. 
And I mount it to vm as a volume. I tested the performance both for SAS and SSD.
This is my current result with iometer.

Test ID

Application

Block Size (Bytes)

Read/Write %

Random/Sequential %

I/O Performance Metrics

iscsi-hdd

iscsi-ssd

Guster,HDD

Gluster,SSD

1

Web File Server

4K

95%/5%

75%/25%

IOPS

195.28

2400.63

208.50

646.75

2

Web File Server

8K

95%/5%

75%/25%

IOPS

193.97

2225.73

179.89

649.93

3

Web File Server

64K

95%/5%

75%/25%

IOPS

180.50

1055.05

158.28

402.15

4

Database Online Transaction Processing

8K

70%/30%

100%/0%

IOPS

163.60

2415.90

132.13

308.46

5

Exchange Email

4K

67%/33%

100%/0%

IOPS

167.03

2685.33

145.92

294.56

6

OS Drive

8K

70%/30%

100%/0%

IOPS

163.60

2407.02

146.11

310.82

7

Decision Support System

1M

100%/0%

100%/0%

IOPS

74.24

207.42

81.14

112.20

8

File Server

8K

90%/10%

75%/25%

IOPS

191.20

2102.32

359.54

526.86

9

Video on Demand

512K

100%/0%

100%/0%

IOPS

100.32

327.19

136.66

162.48

10

Traffic Simulation

8K

50%/50%

75%/25%

IOPS

238.28

1923.91

301.05

297.70

11

Web Server Logging

8K

0%/100%

0%/100%

IOPS

3488.47

3644.33

290.33

282.04

12

SQL Server Logging

64K

0%/100%

0%/100%

IOPS

1423.29

1375.33

182.67

168.64

13

OS Paging

64K

90%/10%

0%/100%

IOPS

1215.74

1211.01

381.17

355.99

14

Media Streaming

64K

98%/2%

0%/100%

IOPS

1350.96

1365.22

457.28

455.49


The issue for GlusterFS I found is

-  It can not retain the nature of SSD and SAS, such as SAS is strong  
for sequential access. SSD is strong for random access.

-  In some case, especially for SSD, the performance is pretty bad.

Thanks,
Cong


From: Donny Davis [mailto:do...@cloudspin.me]
Sent: Thursday, January 15, 2015 10:27 AM
To: Yue, Cong; users@ovirt.org<mailto:users@ovirt.org>
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block storage 
for VMs

Do you have any metrics to give an idea of the difference. I am using NFS right 
now, and I am migrating to Gluster. I have the gluster system up, and I see 
that it seems to provision disks faster than my NFS. I haven't used any real 
measurement tools to get actual metrics, this is all perceived.

Do you have an operational gluster?
Do what are you using right now?

Donny D
cloudspin.me

From: users-boun...@ovirt.org<mailto:users-boun...@ovirt.org> 
[mailto:users-boun...@ovirt.org] On Behalf Of Yue, Cong
Sent: Thursday, January 15, 2015 10:57 AM
To: users@ovirt.org<mailto:users@ovirt.org>
Subject: [ovirt-users] Performance issue for GlusterFS as the block storage for 
VMs

Hi

I have one question about whether GlusterFS is the suitable solution to be used 
as the block storage for VMs.
The failure tolerant and scalability is good for GlusterFS, but in my test, it 
seems the iops is pretty bad. In some blog, it said, it is even with worse 
performance than normal NFS.
Should I use iscsi+drbd for the block storage for VMs.
Can somebody give some advice for this?

Thanks,
Cong


This e-mail message is for the sole use of the intended recipient(s) and may 
contain confidential and privileged information. Any unauthorized review, use, 
disclosure or distribution is prohibited. If you are not the intended 
recipient, please contact the sender by reply e-mail and destroy all copies of 
the original message. If you are the intended recipient, please be advised that 
the content of this message is subject to access, review and disclosure by the 
sender's e-mail System Administrator.


This e-mail message is for the sole use of the intended recipient(s) and may 
contain confidential and privileged information. Any unauthorized review, u

Re: [ovirt-users] Performance issue for GlusterFS as the block storage for VMs

2015-01-15 Thread Donny Davis 
I see. 

 

So have you done any tuning for IO performance or are the configs straight
out of the box. You also said you mounted the volume to a vm. Did you mount
it as gluster or use the built in NFS??

 

Donny

 

From: Yue, Cong [mailto:cong_...@alliedtelesis.com] 
Sent: Thursday, January 15, 2015 11:40 AM
To: Donny Davis; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block
storage for VMs

 

I am using iometer (http://www.iometer.org/) to test the IOPS from one of
the vm.

As for IOPS is more trying to test the performance of block, rather than
real file transfer.

 

In my environment, I am using 10Gbe to make two gluster nodes be replicated.
And I mount it to vm as a volume. I tested the performance both for SAS and
SSD.

This is my current result with iometer. 

 


Test ID

Application

Block Size (Bytes)

Read/Write %

Random/Sequential %

I/O Performance Metrics

iscsi-hdd

iscsi-ssd

Guster,HDD

Gluster,SSD


1

Web File Server

4K

95%/5%

75%/25%

IOPS

195.28

2400.63

208.50

646.75


2

Web File Server

8K

95%/5%

75%/25%

IOPS

193.97

2225.73

179.89

649.93


3

Web File Server

64K

95%/5%

75%/25%

IOPS

180.50

1055.05

158.28

402.15


4

Database Online Transaction Processing

8K

70%/30%

100%/0%

IOPS

163.60

2415.90

132.13

308.46


5

Exchange Email

4K

67%/33%

100%/0%

IOPS

167.03

2685.33

145.92

294.56


6

OS Drive

8K

70%/30%

100%/0%

IOPS

163.60

2407.02

146.11

310.82


7

Decision Support System

1M

100%/0%

100%/0%

IOPS

74.24

207.42

81.14

112.20


8

File Server

8K

90%/10%

75%/25%

IOPS

191.20

2102.32

359.54

526.86


9

Video on Demand

512K

100%/0%

100%/0%

IOPS

100.32

327.19

136.66

162.48


10

Traffic Simulation

8K

50%/50%

75%/25%

IOPS

238.28

1923.91

301.05

297.70


11

Web Server Logging

8K

0%/100%

0%/100%

IOPS

3488.47

3644.33

290.33

282.04


12

SQL Server Logging

64K

0%/100%

0%/100%

IOPS

1423.29

1375.33

182.67

168.64


13

OS Paging

64K

90%/10%

0%/100%

IOPS

1215.74

1211.01

381.17

355.99


14

Media Streaming

64K

98%/2%

0%/100%

IOPS

1350.96

1365.22

457.28

455.49

 

The issue for GlusterFS I found is 

-  It can not retain the nature of SSD and SAS, such as SAS is
strong  for sequential access. SSD is strong for random access.

-  In some case, especially for SSD, the performance is pretty bad.

 

Thanks,

Cong

 

 

From: Donny Davis [mailto:do...@cloudspin.me] 
Sent: Thursday, January 15, 2015 10:27 AM
To: Yue, Cong; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block
storage for VMs

 

Do you have any metrics to give an idea of the difference. I am using NFS
right now, and I am migrating to Gluster. I have the gluster system up, and
I see that it seems to provision disks faster than my NFS. I haven't used
any real measurement tools to get actual metrics, this is all perceived. 

 

Do you have an operational gluster?

Do what are you using right now?

 

Donny D

cloudspin.me

 

From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of
Yue, Cong
Sent: Thursday, January 15, 2015 10:57 AM
To: users@ovirt.org
Subject: [ovirt-users] Performance issue for GlusterFS as the block storage
for VMs

 

Hi 

 

I have one question about whether GlusterFS is the suitable solution to be
used as the block storage for VMs.

The failure tolerant and scalability is good for GlusterFS, but in my test,
it seems the iops is pretty bad. In some blog, it said, it is even with
worse performance than normal NFS.

Should I use iscsi+drbd for the block storage for VMs.

Can somebody give some advice for this? 

 

Thanks,
Cong

 

  _  

This e-mail message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply e-mail and destroy all copies
of the original message. If you are the intended recipient, please be
advised that the content of this message is subject to access, review and
disclosure by the sender's e-mail System Administrator.

 

  _  

This e-mail message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply e-mail and destroy all copies
of the original message. If you are the intended recipient, please be
advised that the content of this message is subject to access, review and
disclosure by the sender's e-mail System Administrator.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance issue for GlusterFS as the block storage for VMs

2015-01-15 Thread Yue, Cong 
I am using iometer (http://www.iometer.org/) to test the IOPS from one of the 
vm.
As for IOPS is more trying to test the performance of block, rather than real 
file transfer.

In my environment, I am using 10Gbe to make two gluster nodes be replicated. 
And I mount it to vm as a volume. I tested the performance both for SAS and SSD.
This is my current result with iometer.

Test ID

Application

Block Size (Bytes)

Read/Write %

Random/Sequential %

I/O Performance Metrics

iscsi-hdd

iscsi-ssd

Guster,HDD

Gluster,SSD

1

Web File Server

4K

95%/5%

75%/25%

IOPS

195.28

2400.63

208.50

646.75

2

Web File Server

8K

95%/5%

75%/25%

IOPS

193.97

2225.73

179.89

649.93

3

Web File Server

64K

95%/5%

75%/25%

IOPS

180.50

1055.05

158.28

402.15

4

Database Online Transaction Processing

8K

70%/30%

100%/0%

IOPS

163.60

2415.90

132.13

308.46

5

Exchange Email

4K

67%/33%

100%/0%

IOPS

167.03

2685.33

145.92

294.56

6

OS Drive

8K

70%/30%

100%/0%

IOPS

163.60

2407.02

146.11

310.82

7

Decision Support System

1M

100%/0%

100%/0%

IOPS

74.24

207.42

81.14

112.20

8

File Server

8K

90%/10%

75%/25%

IOPS

191.20

2102.32

359.54

526.86

9

Video on Demand

512K

100%/0%

100%/0%

IOPS

100.32

327.19

136.66

162.48

10

Traffic Simulation

8K

50%/50%

75%/25%

IOPS

238.28

1923.91

301.05

297.70

11

Web Server Logging

8K

0%/100%

0%/100%

IOPS

3488.47

3644.33

290.33

282.04

12

SQL Server Logging

64K

0%/100%

0%/100%

IOPS

1423.29

1375.33

182.67

168.64

13

OS Paging

64K

90%/10%

0%/100%

IOPS

1215.74

1211.01

381.17

355.99

14

Media Streaming

64K

98%/2%

0%/100%

IOPS

1350.96

1365.22

457.28

455.49


The issue for GlusterFS I found is

-  It can not retain the nature of SSD and SAS, such as SAS is strong  
for sequential access. SSD is strong for random access.

-  In some case, especially for SSD, the performance is pretty bad.

Thanks,
Cong


From: Donny Davis [mailto:do...@cloudspin.me]
Sent: Thursday, January 15, 2015 10:27 AM
To: Yue, Cong; users@ovirt.org
Subject: RE: [ovirt-users] Performance issue for GlusterFS as the block storage 
for VMs

Do you have any metrics to give an idea of the difference. I am using NFS right 
now, and I am migrating to Gluster. I have the gluster system up, and I see 
that it seems to provision disks faster than my NFS. I haven't used any real 
measurement tools to get actual metrics, this is all perceived.

Do you have an operational gluster?
Do what are you using right now?

Donny D
cloudspin.me

From: users-boun...@ovirt.org<mailto:users-boun...@ovirt.org> 
[mailto:users-boun...@ovirt.org] On Behalf Of Yue, Cong
Sent: Thursday, January 15, 2015 10:57 AM
To: users@ovirt.org<mailto:users@ovirt.org>
Subject: [ovirt-users] Performance issue for GlusterFS as the block storage for 
VMs

Hi

I have one question about whether GlusterFS is the suitable solution to be used 
as the block storage for VMs.
The failure tolerant and scalability is good for GlusterFS, but in my test, it 
seems the iops is pretty bad. In some blog, it said, it is even with worse 
performance than normal NFS.
Should I use iscsi+drbd for the block storage for VMs.
Can somebody give some advice for this?

Thanks,
Cong


This e-mail message is for the sole use of the intended recipient(s) and may 
contain confidential and privileged information. Any unauthorized review, use, 
disclosure or distribution is prohibited. If you are not the intended 
recipient, please contact the sender by reply e-mail and destroy all copies of 
the original message. If you are the intended recipient, please be advised that 
the content of this message is subject to access, review and disclosure by the 
sender's e-mail System Administrator.


This e-mail message is for the sole use of the intended recipient(s) and may 
contain confidential and privileged information. Any unauthorized review, use, 
disclosure or distribution is prohibited. If you are not the intended 
recipient, please contact the sender by reply e-mail and destroy all copies of 
the original message. If you are the intended recipient, please be advised that 
the content of this message is subject to access, review and disclosure by the 
sender's e-mail System Administrator.
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Performance issue for GlusterFS as the block storage for VMs

2015-01-15 Thread Donny Davis 
Do you have any metrics to give an idea of the difference. I am using NFS
right now, and I am migrating to Gluster. I have the gluster system up, and
I see that it seems to provision disks faster than my NFS. I haven't used
any real measurement tools to get actual metrics, this is all perceived. 

 

Do you have an operational gluster?

Do what are you using right now?

 

Donny D

cloudspin.me

 

From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of
Yue, Cong
Sent: Thursday, January 15, 2015 10:57 AM
To: users@ovirt.org
Subject: [ovirt-users] Performance issue for GlusterFS as the block storage
for VMs

 

Hi 

 

I have one question about whether GlusterFS is the suitable solution to be
used as the block storage for VMs.

The failure tolerant and scalability is good for GlusterFS, but in my test,
it seems the iops is pretty bad. In some blog, it said, it is even with
worse performance than normal NFS.

Should I use iscsi+drbd for the block storage for VMs.

Can somebody give some advice for this? 

 

Thanks,
Cong

 

  _  

This e-mail message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply e-mail and destroy all copies
of the original message. If you are the intended recipient, please be
advised that the content of this message is subject to access, review and
disclosure by the sender's e-mail System Administrator.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Performance issue for GlusterFS as the block storage for VMs

2015-01-15 Thread Yue, Cong 
Hi

I have one question about whether GlusterFS is the suitable solution to be used 
as the block storage for VMs.
The failure tolerant and scalability is good for GlusterFS, but in my test, it 
seems the iops is pretty bad. In some blog, it said, it is even with worse 
performance than normal NFS.
Should I use iscsi+drbd for the block storage for VMs.
Can somebody give some advice for this?

Thanks,
Cong


This e-mail message is for the sole use of the intended recipient(s) and may 
contain confidential and privileged information. Any unauthorized review, use, 
disclosure or distribution is prohibited. If you are not the intended 
recipient, please contact the sender by reply e-mail and destroy all copies of 
the original message. If you are the intended recipient, please be advised that 
the content of this message is subject to access, review and disclosure by the 
sender's e-mail System Administrator.
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users