Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Kandalf ®]

Yes, will write to both gluster nodes simultaneously. 
 


 On Saturday, October 17, 2015 3:14 AM, Lindsay Mathieson 
 wrote:
   

 
On 17 October 2015 at 09:49, Kandalf ®  wrote:

Yes, now I install the 2 new Slackware machines and I want to setup gluster + 
iscsi. I hope that will work as I expected. Do you know to be some issues if I 
use multipath to the both gluster servers from esxi?

Not sure, bit outside my expertise there :) Would multipath just present you 
with fail over or would it actually involve esxi writing to the vm image on 
both gluster nodes simultaneously?

If the later, vest answered by the devs.


-- 
Lindsay

  ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Alastair Neil]

is that a typo 120mb/s?  Is this volume replica 2 or distributed or both?


On 16 October 2015 at 08:05, Kandalf ®  wrote:

> Hi,
>
> I have 2 server with centos 7 and I want to make a replicate storage
> cluster for the esxi.
> In both server I have 4 disk in raid 5 mdadm. The write speed on that
> device is ~250-300MB/s all the time.
> I create xfs file system on the mdadm and I exported that gluster volume
> via native fuse nfs v3 to the esxi.
> Read seed is great but when I try to write to the guest vms I see
> 20-40MB/s only. I try the test also with gluster distributed with only one
> brick to one server. If I take one dd write from esxi to the cluster I
> receive 120mb/s the full ethernet link speed. I also try to mount to linux
> that volumes and try dd write and the write speed is 120mb/s.
> But if I try to mount an raw file image with losetup, or use vmdk files,
> and if I write to them, the speed is 20-40MB/s and I see 12 load on linux.
> Can someone help me?
>
> Thanks!
>
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Alastair Neil]

I'm getting confused, 120 mb/s is less that 20 MB/s  unless you mean
120MB/s?

On 16 October 2015 at 16:26, Kandalf ®  wrote:

> No, is real speed. From any computer linux or esxi if I write via NFS with
> dd utility I have this speed. But if I try to write to one raw file like
> esxi does in vmdk, than the speed drops to 20-40mb/s. So my issue is low
> speed when I write into raw image file (using losetup and DD on linux , or
> esxi vmdk - vm guest)
>
>
>
> On Friday, October 16, 2015 8:31 PM, Alastair Neil 
> wrote:
>
>
> is that a typo 120mb/s?  Is this volume replica 2 or distributed or both?
>
>
> On 16 October 2015 at 08:05, Kandalf ®  wrote:
>
> Hi,
>
> I have 2 server with centos 7 and I want to make a replicate storage
> cluster for the esxi.
> In both server I have 4 disk in raid 5 mdadm. The write speed on that
> device is ~250-300MB/s all the time.
> I create xfs file system on the mdadm and I exported that gluster volume
> via native fuse nfs v3 to the esxi.
> Read seed is great but when I try to write to the guest vms I see
> 20-40MB/s only. I try the test also with gluster distributed with only one
> brick to one server. If I take one dd write from esxi to the cluster I
> receive 120mb/s the full ethernet link speed. I also try to mount to linux
> that volumes and try dd write and the write speed is 120mb/s.
> But if I try to mount an raw file image with losetup, or use vmdk files,
> and if I write to them, the speed is 20-40MB/s and I see 12 load on linux.
> Can someone help me?
>
> Thanks!
>
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Kandalf ®]

No, is real speed. From any computer linux or esxi if I write via NFS with dd 
utility I have this speed. But if I try to write to one raw file like esxi does 
in vmdk, than the speed drops to 20-40mb/s. So my issue is low speed when I 
write into raw image file (using losetup and DD on linux , or esxi vmdk - vm 
guest)
 


 On Friday, October 16, 2015 8:31 PM, Alastair Neil  
wrote:
   

 is that a typo 120mb/s?  Is this volume replica 2 or distributed or both?

On 16 October 2015 at 08:05, Kandalf ®  wrote:

Hi,
I have 2 server with centos 7 and I want to make a replicate storage cluster 
for the esxi.In both server I have 4 disk in raid 5 mdadm. The write speed on 
that device is ~250-300MB/s all the time.I create xfs file system on the mdadm 
and I exported that gluster volume via native fuse nfs v3 to the esxi.Read seed 
is great but when I try to write to the guest vms I see 20-40MB/s only. I try 
the test also with gluster distributed with only one brick to one server. If I 
take one dd write from esxi to the cluster I receive 120mb/s the full ethernet 
link speed. I also try to mount to linux that volumes and try dd write and the 
write speed is 120mb/s.But if I try to mount an raw file image with losetup, or 
use vmdk files, and if I write to them, the speed is 20-40MB/s and I see 12 
load on linux.Can someone help me?
Thanks!



___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users




  ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Kandalf ®]

If I write with DD in linux locally to the same cluster node... or via network 
I have 120MB/s. the full speed of te 1 Gbps.But if I write to an RAW Image 
File... like vmdk in vmware... I have average of 20-40MB/s only.
 


 On Friday, October 16, 2015 11:31 PM, Alastair Neil 
 wrote:
   

 I'm getting confused, 120 mb/s is less that 20 MB/s  unless you mean 120MB/s?
On 16 October 2015 at 16:26, Kandalf ®  wrote:

No, is real speed. From any computer linux or esxi if I write via NFS with dd 
utility I have this speed. But if I try to write to one raw file like esxi does 
in vmdk, than the speed drops to 20-40mb/s. So my issue is low speed when I 
write into raw image file (using losetup and DD on linux , or esxi vmdk - vm 
guest)
 


 On Friday, October 16, 2015 8:31 PM, Alastair Neil  
wrote:
   

 is that a typo 120mb/s?  Is this volume replica 2 or distributed or both?

On 16 October 2015 at 08:05, Kandalf ®  wrote:

Hi,
I have 2 server with centos 7 and I want to make a replicate storage cluster 
for the esxi.In both server I have 4 disk in raid 5 mdadm. The write speed on 
that device is ~250-300MB/s all the time.I create xfs file system on the mdadm 
and I exported that gluster volume via native fuse nfs v3 to the esxi.Read seed 
is great but when I try to write to the guest vms I see 20-40MB/s only. I try 
the test also with gluster distributed with only one brick to one server. If I 
take one dd write from esxi to the cluster I receive 120mb/s the full ethernet 
link speed. I also try to mount to linux that volumes and try dd write and the 
write speed is 120mb/s.But if I try to mount an raw file image with losetup, or 
use vmdk files, and if I write to them, the speed is 20-40MB/s and I see 12 
load on linux.Can someone help me?
Thanks!



___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users




   



  ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Kandalf ®]

I also tried more gluster version but the same situation.If I use iscsi for 
example to export a raw image... I have 120MB/s write speed.
 


 On Friday, October 16, 2015 11:42 PM, Kandalf ®  wrote:
   

 If I write with DD in linux locally to the same cluster node... or via network 
I have 120MB/s. the full speed of te 1 Gbps.But if I write to an RAW Image 
File... like vmdk in vmware... I have average of 20-40MB/s only.
 


 On Friday, October 16, 2015 11:31 PM, Alastair Neil 
 wrote:
   

 I'm getting confused, 120 mb/s is less that 20 MB/s  unless you mean 120MB/s?
On 16 October 2015 at 16:26, Kandalf ®  wrote:

No, is real speed. From any computer linux or esxi if I write via NFS with dd 
utility I have this speed. But if I try to write to one raw file like esxi does 
in vmdk, than the speed drops to 20-40mb/s. So my issue is low speed when I 
write into raw image file (using losetup and DD on linux , or esxi vmdk - vm 
guest)
 


 On Friday, October 16, 2015 8:31 PM, Alastair Neil  
wrote:
   

 is that a typo 120mb/s?  Is this volume replica 2 or distributed or both?

On 16 October 2015 at 08:05, Kandalf ®  wrote:

Hi,
I have 2 server with centos 7 and I want to make a replicate storage cluster 
for the esxi.In both server I have 4 disk in raid 5 mdadm. The write speed on 
that device is ~250-300MB/s all the time.I create xfs file system on the mdadm 
and I exported that gluster volume via native fuse nfs v3 to the esxi.Read seed 
is great but when I try to write to the guest vms I see 20-40MB/s only. I try 
the test also with gluster distributed with only one brick to one server. If I 
take one dd write from esxi to the cluster I receive 120mb/s the full ethernet 
link speed. I also try to mount to linux that volumes and try dd write and the 
write speed is 120mb/s.But if I try to mount an raw file image with losetup, or 
use vmdk files, and if I write to them, the speed is 20-40MB/s and I see 12 
load on linux.Can someone help me?
Thanks!



___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users




   



   

  ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Alastair Neil]

I believe if this is replica 3 then you would expect 40 MB/s, as the client
has to write to 3 bricks.  You only mentioned 2 servers so I assume this is
at most replica 2, so 60 MB/s would be what I would expect.  This is still
1.5-3x the numbers you are reporting though.

On 16 October 2015 at 16:43, Kandalf ®  wrote:

> I also tried more gluster version but the same situation.If I use iscsi
> for example to export a raw image... I have 120MB/s write speed.
>
>
>
> On Friday, October 16, 2015 11:42 PM, Kandalf ®  wrote:
>
>
> If I write with DD in linux locally to the same cluster node... or via
> network I have 120MB/s. the full speed of te 1 Gbps.
> But if I write to an RAW Image File... like vmdk in vmware... I have
> average of 20-40MB/s only.
>
>
>
> On Friday, October 16, 2015 11:31 PM, Alastair Neil 
> wrote:
>
>
> I'm getting confused, 120 mb/s is less that 20 MB/s  unless you mean
> 120MB/s?
>
> On 16 October 2015 at 16:26, Kandalf ®  wrote:
>
> No, is real speed. From any computer linux or esxi if I write via NFS with
> dd utility I have this speed. But if I try to write to one raw file like
> esxi does in vmdk, than the speed drops to 20-40mb/s. So my issue is low
> speed when I write into raw image file (using losetup and DD on linux , or
> esxi vmdk - vm guest)
>
>
>
> On Friday, October 16, 2015 8:31 PM, Alastair Neil 
> wrote:
>
>
> is that a typo 120mb/s?  Is this volume replica 2 or distributed or both?
>
>
> On 16 October 2015 at 08:05, Kandalf ®  wrote:
>
> Hi,
>
> I have 2 server with centos 7 and I want to make a replicate storage
> cluster for the esxi.
> In both server I have 4 disk in raid 5 mdadm. The write speed on that
> device is ~250-300MB/s all the time.
> I create xfs file system on the mdadm and I exported that gluster volume
> via native fuse nfs v3 to the esxi.
> Read seed is great but when I try to write to the guest vms I see
> 20-40MB/s only. I try the test also with gluster distributed with only one
> brick to one server. If I take one dd write from esxi to the cluster I
> receive 120mb/s the full ethernet link speed. I also try to mount to linux
> that volumes and try dd write and the write speed is 120mb/s.
> But if I try to mount an raw file image with losetup, or use vmdk files,
> and if I write to them, the speed is 20-40MB/s and I see 12 load on linux.
> Can someone help me?
>
> Thanks!
>
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
>
>
>
>
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Kandalf ®]

Hi,
I have 2 server with centos 7 and I want to make a replicate storage cluster 
for the esxi.In both server I have 4 disk in raid 5 mdadm. The write speed on 
that device is ~250-300MB/s all the time.I create xfs file system on the mdadm 
and I exported that gluster volume via native fuse nfs v3 to the esxi.Read seed 
is great but when I try to write to the guest vms I see 20-40MB/s only. I try 
the test also with gluster distributed with only one brick to one server. If I 
take one dd write from esxi to the cluster I receive 120mb/s the full ethernet 
link speed. I also try to mount to linux that volumes and try dd write and the 
write speed is 120mb/s.But if I try to mount an raw file image with losetup, or 
use vmdk files, and if I write to them, the speed is 20-40MB/s and I see 12 
load on linux.Can someone help me?
Thanks!


___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Kandalf ®]

I use Vsphere and I have replica 2. I share the image via gluster nfs-fuse.I 
have 2 dedicate LANs one to esxi and another for gluster cluster replication 
thats why I have 120MB/s when I create a new file onto the gluster cluster. But 
when I try to use that file like vmdk, the speed drops to ~20-40MB/s.
 


 On Saturday, October 17, 2015 1:12 AM, Lindsay Mathieson 
 wrote:
   

 
On 16 October 2015 at 22:05, Kandalf ®  wrote:

But if I try to mount an raw file image with losetup, or use vmdk files, and if 
I write to them, the speed is 20-40MB/s and I see 12 load on linux.

What VM hosting are you using? KVM/Qemu? vSphere?

How is the image shared out? gfapi? nfs?


-- 
Lindsay

  ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Lindsay Mathieson]

>From memory, nfs doesn't play very well with vSphere. Are you able to try
iSCSI over gluster?

http://gluster.readthedocs.org/en/latest/Administrator%20Guide/GlusterFS%20iSCSI/

On 17 October 2015 at 08:56, Kandalf ®  wrote:

> I use Vsphere and I have replica 2. I share the image via gluster nfs-fuse.
> I have 2 dedicate LANs one to esxi and another for gluster cluster
> replication thats why I have 120MB/s when I create a new file onto the
> gluster cluster. But when I try to use that file like vmdk, the speed drops
> to ~20-40MB/s.
>
>
>
> On Saturday, October 17, 2015 1:12 AM, Lindsay Mathieson <
> lindsay.mathie...@gmail.com> wrote:
>
>
>
> On 16 October 2015 at 22:05, Kandalf ®  wrote:
>
> But if I try to mount an raw file image with losetup, or use vmdk files,
> and if I write to them, the speed is 20-40MB/s and I see 12 load on linux.
>
>
> What VM hosting are you using? KVM/Qemu? vSphere?
>
> How is the image shared out? gfapi? nfs?
>
>
> --
> Lindsay
>
>
>


-- 
Lindsay
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Lindsay Mathieson]

On 17 October 2015 at 09:49, Kandalf ®  wrote:

> Yes, now I install the 2 new Slackware machines and I want to setup
> gluster + iscsi. I hope that will work as I expected. Do you know to be
> some issues if I use multipath to the both gluster servers from esxi?
>

Not sure, bit outside my expertise there :) Would multipath just present
you with fail over or would it actually involve esxi writing to the vm
image on both gluster nodes simultaneously?

If the later, vest answered by the devs.


-- 
Lindsay
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Lindsay Mathieson]

On 16 October 2015 at 22:05, Kandalf ®  wrote:

> But if I try to mount an raw file image with losetup, or use vmdk files,
> and if I write to them, the speed is 20-40MB/s and I see 12 load on linux.


What VM hosting are you using? KVM/Qemu? vSphere?

How is the image shared out? gfapi? nfs?


-- 
Lindsay
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster 3.7.5 low speed on write and heavy load

[Kandalf ®]

Yes, now I install the 2 new Slackware machines and I want to setup gluster + 
iscsi. I hope that will work as I expected. Do you know to be some issues if I 
use multipath to the both gluster servers from esxi?
 


 On Saturday, October 17, 2015 2:17 AM, Lindsay Mathieson 
 wrote:
   

 From memory, nfs doesn't play very well with vSphere. Are you able to try 
iSCSI over gluster?

http://gluster.readthedocs.org/en/latest/Administrator%20Guide/GlusterFS%20iSCSI/

On 17 October 2015 at 08:56, Kandalf ®  wrote:

I use Vsphere and I have replica 2. I share the image via gluster nfs-fuse.I 
have 2 dedicate LANs one to esxi and another for gluster cluster replication 
thats why I have 120MB/s when I create a new file onto the gluster cluster. But 
when I try to use that file like vmdk, the speed drops to ~20-40MB/s.
 


 On Saturday, October 17, 2015 1:12 AM, Lindsay Mathieson 
 wrote:
   

 
On 16 October 2015 at 22:05, Kandalf ®  wrote:

But if I try to mount an raw file image with losetup, or use vmdk files, and if 
I write to them, the speed is 20-40MB/s and I see 12 load on linux.

What VM hosting are you using? KVM/Qemu? vSphere?

How is the image shared out? gfapi? nfs?


-- 
Lindsay

   



-- 
Lindsay

  ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users