Re: [Gluster-users] Slow write times to gluster disk

On 6/30/18, Raghavendra Gowdappa wrote: > On Fri, Jun 29, 2018 at 10:38 PM, Pat Haley wrote: > >> >> Hi Raghavendra, >> >> We ran the tests (write tests) and I copied the log files for both the >> server and the client to http://mseas.mit.edu/download/ >> phaley/GlusterUsers/2018/Jun29/ . Is

Re: [Gluster-users] Slow write times to gluster disk

On Fri, Jul 13, 2018 at 5:00 AM, Pat Haley wrote: > > Hi Raghavendra, > > We were wondering if you have had a chance to look at this again, and if > so, did you have any further suggestions? > Sorry Pat. Too much of work :). I'll be working on a patch today to make sure read-ahead doesn't flush

Re: [Gluster-users] Slow write times to gluster disk

Hi Raghavendra, We were wondering if you have had a chance to look at this again, and if so, did you have any further suggestions? Thanks Pat On 07/06/2018 01:27 AM, Raghavendra Gowdappa wrote: On Fri, Jul 6, 2018 at 5:29 AM, Pat Haley > wrote: Hi

Re: [Gluster-users] Slow write times to gluster disk

On Fri, Jul 6, 2018 at 5:29 AM, Pat Haley wrote: > > Hi Raghavendra, > > Our technician may have some time to look at this issue tomorrow. Are > there any tests that you'd like to see? > Sorry. I've been busy with other things and was away from work for couple of days. It'll take me another 2

Re: [Gluster-users] Slow write times to gluster disk

Hi Raghavendra, Our technician may have some time to look at this issue tomorrow. Are there any tests that you'd like to see? Thanks Pat On 06/29/2018 11:25 PM, Raghavendra Gowdappa wrote: On Fri, Jun 29, 2018 at 10:38 PM, Pat Haley > wrote: Hi

Re: [Gluster-users] Slow write times to gluster disk

On Fri, Jun 29, 2018 at 10:38 PM, Pat Haley wrote: > > Hi Raghavendra, > > We ran the tests (write tests) and I copied the log files for both the > server and the client to http://mseas.mit.edu/download/ > phaley/GlusterUsers/2018/Jun29/ . Is there any additional trace > information you need?

Re: [Gluster-users] Slow write times to gluster disk

Hi Raghavendra, We ran the tests (write tests) and I copied the log files for both the server and the client to http://mseas.mit.edu/download/phaley/GlusterUsers/2018/Jun29/ .  Is there any additional trace information you need?  (If so, where should I look for it?) Also the volume

Re: [Gluster-users] Slow write times to gluster disk

On Fri, Jun 29, 2018 at 8:24 PM, Pat Haley wrote: > > Hi Raghavendra, > > Our technician was able to try the manual setting today. He found that > our upper limit for performance.md-cache-timeout was 60 not 600, so he > used that value, along with the network.inode-lru-limit=5. > > The

Re: [Gluster-users] Slow write times to gluster disk

Hi Raghavendra, Our technician was able to try the manual setting today.  He found that our upper limit for performance.md-cache-timeout was 60 not 600, so he used that value, along with the network.inode-lru-limit=5. The result was another small (~1%) increase in speed.  Does this

Re: [Gluster-users] Slow write times to gluster disk

On Tue, Jun 26, 2018 at 3:21 AM, Pat Haley wrote: > > Hi Raghavendra, > > Setting the performance.write-behind off had a small improvement on the > write speed (~3%), > > We were unable to turn on "group metadata-cache". When we try get errors > like > > # gluster volume set data-volume group

Re: [Gluster-users] Slow write times to gluster disk

Hi Raghavendra, Setting the performance.write-behind off had a small improvement on the write speed (~3%), We were unable to turn on "group metadata-cache".  When we try get errors like # gluster volume set data-volume group metadata-cache '/var/lib/glusterd/groups/metadata-cache' file

Re: [Gluster-users] Slow write times to gluster disk

On Thu, Jun 21, 2018 at 8:41 PM, Pat Haley wrote: > > Hi Raghavendra, > > Thanks for the suggestions. Our technician will be in on Monday. We'll > test then and let you know the results. > > One question I have, is the "group metadata-cache" option supposed to > directly impact the performance

Re: [Gluster-users] Slow write times to gluster disk

Hi Raghavendra, Thanks for the suggestions.  Our technician will be in on Monday. We'll test then and let you know the results. One question I have, is the "group metadata-cache" option supposed to directly impact the performance or is it to help collect data? If the latter, where will the

Re: [Gluster-users] Slow write times to gluster disk

On Thu, Jun 21, 2018 at 10:24 AM, Raghavendra Gowdappa wrote: > For the case of writes to glusterfs mount, > > I saw in earlier conversations that there are too many lookups, but small > number of writes. Since writes cached in write-behind would invalidate > metadata cache, lookups won't be

Re: [Gluster-users] Slow write times to gluster disk

Please note that these suggestions are for native fuse mount. On Thu, Jun 21, 2018 at 10:24 AM, Raghavendra Gowdappa wrote: > For the case of writes to glusterfs mount, > > I saw in earlier conversations that there are too many lookups, but small > number of writes. Since writes cached in

Re: [Gluster-users] Slow write times to gluster disk

For the case of writes to glusterfs mount, I saw in earlier conversations that there are too many lookups, but small number of writes. Since writes cached in write-behind would invalidate metadata cache, lookups won't be absorbed by md-cache. I am wondering what would results look like if we turn

Re: [Gluster-users] Slow write times to gluster disk

On Thu, Jun 21, 2018 at 8:32 AM, Raghavendra Gowdappa wrote: > For the case of reading from Glusterfs mount, read-ahead should help. > However, we've known issues with read-ahead[1][2]. To work around these, > can you try with, > > 1. Turn off performance.open-behind > #gluster volume set

Re: [Gluster-users] Slow write times to gluster disk

For the case of reading from Glusterfs mount, read-ahead should help. However, we've known issues with read-ahead[1][2]. To work around these, can you try with, 1. Turn off performance.open-behind #gluster volume set performance.open-behind off 2. enable group meta metadata-cache # gluster

[Gluster-users] Slow write times to gluster disk

Hi, We were recently revisiting our problems with the slowness of gluster writes (http://lists.gluster.org/pipermail/gluster-users/2017-April/030529.html). Specifically we were testing the suggestions in a recent post

Re: [Gluster-users] Slow write times to gluster disk

_ From: Soumya Koduri <skod...@redhat.com> Sent: Tuesday, August 8, 2017 1:37 AM To: Pat Haley; Niels de Vos Cc: gluster-users@gluster.org; Pranith Kumar Karampuri; Ben Turner; Ravishankar N; Raghavendra Gowdappa; Niels de Vos; Steve Postma Subject: Re: [Gluster-users] Slow write times to

Re: [Gluster-users] Slow write times to gluster disk

.com>, "Ravishankar N" > <ravishan...@redhat.com>, "Raghavendra Gowdappa" > <rgowd...@redhat.com>, "Niels de Vos" <nde...@redhat.com>, "Steve Postma" > <spos...@ztechnet.com> > Sent: Monday, August 7, 2017 9:52:

Re: [Gluster-users] Slow write times to gluster disk

Hi Soumya, We just had the opportunity to try the option of disabling the kernel-NFS and restarting glusterd to start gNFS. However the gluster demon crashes immediately on startup. What additional information besides what we provide below would help debugging this? Thanks, Pat

Re: [Gluster-users] Slow write times to gluster disk

On 07/14/2017 06:40 AM, Pat Haley wrote: Hi Soumya, I just noticed some of the notes at the bottom. In particular * Till glusterfs-3.7, gluster-NFS (gNFS) gets enabled by default. The only requirement is that kernel-NFS has to be disabled for gluster-NFS to come up. Please

Re: [Gluster-users] Slow write times to gluster disk

Hi Soumya, I just noticed some of the notes at the bottom. In particular * Till glusterfs-3.7, gluster-NFS (gNFS) gets enabled by default. The only requirement is that kernel-NFS has to be disabled for gluster-NFS to come up. Please disable kernel-NFS server and restart glusterd to

Re: [Gluster-users] Slow write times to gluster disk

Hi, On 07/07/2017 06:16 AM, Pat Haley wrote: Hi All, A follow-up question. I've been looking at various pages on nfs-ganesha & gluster. Is there a version of nfs-ganesha that is recommended for use with glusterfs 3.7.11 built on Apr 27 2016 14:09:22 CentOS release 6.8 (Final) For

Re: [Gluster-users] Slow write times to gluster disk

Hi All, A follow-up question. I've been looking at various pages on nfs-ganesha & gluster. Is there a version of nfs-ganesha that is recommended for use with glusterfs 3.7.11 built on Apr 27 2016 14:09:22 CentOS release 6.8 (Final) Thanks Pat On 07/05/2017 11:36 AM, Pat Haley wrote:

Re: [Gluster-users] Slow write times to gluster disk

Hi Soumya, (1) In http://mseas.mit.edu/download/phaley/GlusterUsers/TestNFSmount/ I've placed the following 2 log files etc-glusterfs-glusterd.vol.log gdata.log The first has repeated messages about nfs disconnects. The second had the .log name (but not much information). (2) About the

Re: [Gluster-users] Slow write times to gluster disk

On 07/03/2017 09:01 PM, Pat Haley wrote: Hi Soumya, When I originally did the tests I ran tcpdump on the client. I have rerun the tests, doing tcpdump on the server tcpdump -i any -nnSs 0 host 172.16.1.121 -w /root/capture_nfsfail.pcap The results are in the same place

Re: [Gluster-users] Slow write times to gluster disk

Hi Soumya, When I originally did the tests I ran tcpdump on the client. I have rerun the tests, doing tcpdump on the server tcpdump -i any -nnSs 0 host 172.16.1.121 -w /root/capture_nfsfail.pcap The results are in the same place

Re: [Gluster-users] Slow write times to gluster disk

On 06/30/2017 07:56 PM, Pat Haley wrote: Hi, I was wondering if there were any additional test we could perform to help debug the group write-permissions issue? Sorry for the delay. Please find response inline -- Thanks Pat On 06/27/2017 12:29 PM, Pat Haley wrote: Hi Soumya, One

Re: [Gluster-users] Slow write times to gluster disk

Hi, I was wondering if there were any additional test we could perform to help debug the group write-permissions issue? Thanks Pat On 06/27/2017 12:29 PM, Pat Haley wrote: Hi Soumya, One example, we have a common working directory dri_fleat in the gluster volume drwxrwsr-x 22 root

Re: [Gluster-users] Slow write times to gluster disk

Hi Soumya, One example, we have a common working directory dri_fleat in the gluster volume drwxrwsr-x 22 root dri_fleat 4.0K May 1 15:14 dri_fleat my user (phaley) does not own that directory but is a member of the group dri_fleat and should have write permissions. When I go to the

Re: [Gluster-users] Slow write times to gluster disk

ave > >>> /xfs-mount/gluster-brick, if you have something similar just mkdir on the > >>> XFS -> /xfs-mount/my-test-dir. Inside the test dir run: > >>> > >>> If you are focusing on a write workload run: > >>> > >>> # dd

Re: [Gluster-users] Slow write times to gluster disk

On 06/27/2017 10:17 AM, Pranith Kumar Karampuri wrote: The only problem with using gluster mounted via NFS is that it does not respect the group write permissions which we need. We have an exercise coming up in the a couple of weeks. It seems to me that in order to improve our write times

Re: [Gluster-users] Slow write times to gluster disk

mount/file bs=1024k count=1 conv=fdatasync >>> >>> If you are focusing on a read workload run: >>> >>> # echo 3 > /proc/sys/vm/drop_caches >>> # dd if=/gluster-mount/file of=/dev/null bs=1024k count=1 >>> >>> ** MA

Re: [Gluster-users] Slow write times to gluster disk

ve an apples to apples comparison here. -b - Original Message - From: "Pat Haley"<pha...@mit.edu> <mailto:pha...@mit.edu> To: "Ben Turner"<btur...@redhat.com> <mailto:btur...@redhat.com> Sent: Monday, June 12, 201

Re: [Gluster-users] Slow write times to gluster disk

run again on both at the same time. While this is running gather iostat for >> me: >> >> # iostat -c -m -x 1 > iostat-$(hostname).txt >> >> Lets see how the back end performs on both servers while capturing iostat, >> then see how the same workload / data

Re: [Gluster-users] Slow write times to gluster disk

e same workload / data looks on gluster. > > -Last thing, when you run your kernel NFS tests are you using the same > filesystem / storage you are using for the gluster bricks? I want to be sure > we have an apples to apples comparison here. > > -b > > > > - Original

Re: [Gluster-users] Slow write times to gluster disk

we have an apples to apples comparison here. -b - Original Message - From: "Pat Haley"<pha...@mit.edu> To: "Ben Turner"<btur...@redhat.com> Sent: Monday, June 12, 2017 5:18:07 PM Subject: Re: [Gluster-users] Slow write times to gluster disk Hi Ben

Re: [Gluster-users] Slow write times to gluster disk

e gluster bricks? I want to be sure we have an apples to apples comparison here. -b - Original Message - From: "Pat Haley" <pha...@mit.edu> To: "Ben Turner" <btur...@redhat.com> Sent: Monday, June 12, 2017 5:18:07 PM Subject: Re: [Gluster-users] Slow wr

Re: [Gluster-users] Slow write times to gluster disk

luster.org, > "Steve Postma" <spos...@ztechnet.com> > Sent: Monday, June 12, 2017 2:35:41 PM > Subject: Re: [Gluster-users] Slow write times to gluster disk > > > Hi Guys, > > I was wondering what our next steps should be to solve the slow write times

Re: [Gluster-users] Slow write times to gluster disk

han...@redhat.com>, gluster-users@gluster.org, "Steve Postma" <spos...@ztechnet.com>, "Ben Turner" <btur...@redhat.com> Sent: Tuesday, May 30, 2017 9:40:34 PM Subject: Re: [Gluster-users] Slow write times to gluster disk Hi Pranith, The "dd" comm

Re: [Gluster-users] Slow write times to gluster disk

quot; <pkara...@redhat.com> > Cc: "Ravishankar N" <ravishan...@redhat.com>, gluster-users@gluster.org, > "Steve Postma" <spos...@ztechnet.com>, "Ben > Turner" <btur...@redhat.com> > Sent: Tuesday, May 30, 2017 9:40:34 PM > Su

Re: [Gluster-users] Slow write times to gluster disk

Hi Soumya, What pattern should we be trying to view with the tcpump? Is a one minute capture of a copy operation sufficient or are you looking for something else? Pat On 05/31/2017 06:56 AM, Soumya Koduri wrote: On 05/31/2017 07:24 AM, Pranith Kumar Karampuri wrote: Thanks this is

Re: [Gluster-users] Slow write times to gluster disk

On 05/31/2017 07:24 AM, Pranith Kumar Karampuri wrote: Thanks this is good information. +Soumya Soumya, We are trying to find why kNFS is performing way better than plain distribute glusterfs+fuse. What information do you think will benefit us to compare the operations with kNFS vs

Re: [Gluster-users] Slow write times to gluster disk

Thanks this is good information. +Soumya Soumya, We are trying to find why kNFS is performing way better than plain distribute glusterfs+fuse. What information do you think will benefit us to compare the operations with kNFS vs gluster+fuse? We already have profile output from fuse. On

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, The "dd" command was: dd if=/dev/zero count=4096 bs=1048576 of=zeros.txt conv=sync There were 2 instances where dd reported 22 seconds. The output from the dd tests are in http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/dd_testvol_gluster.txt Pat On 05/30/2017

Re: [Gluster-users] Slow write times to gluster disk

Pat, What is the command you used? As per the following output, it seems like at least one write operation took 16 seconds. Which is really bad. 96.391165.10 us 89.00 us *16487014.00 us* 393212 WRITE On Tue, May 30, 2017 at 10:36 PM, Pat Haley

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, I ran the same 'dd' test both in the gluster test volume and in the .glusterfs directory of each brick. The median results (12 dd trials in each test) are similar to before * gluster test volume: 586.5 MB/s * bricks (in .glusterfs): 1.4 GB/s The profile for the gluster

Re: [Gluster-users] Slow write times to gluster disk

Let's start with the same 'dd' test we were testing with to see, what the numbers are. Please provide profile numbers for the same. From there on we will start tuning the volume to see what we can do. On Tue, May 30, 2017 at 9:16 PM, Pat Haley wrote: > > Hi Pranith, > > Thanks

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, Thanks for the tip. We now have the gluster volume mounted under /home. What tests do you recommend we run? Thanks Pat On 05/17/2017 05:01 AM, Pranith Kumar Karampuri wrote: On Tue, May 16, 2017 at 9:20 PM, Pat Haley > wrote:

Re: [Gluster-users] Slow write times to gluster disk

On Wed, May 17, 2017 at 9:54 PM, Joe Julian wrote: > On 05/17/17 02:02, Pranith Kumar Karampuri wrote: > > On Tue, May 16, 2017 at 9:38 PM, Joe Julian wrote: > >> On 04/13/17 23:50, Pranith Kumar Karampuri wrote: >> >> >> >> On Sat, Apr 8, 2017 at

Re: [Gluster-users] Slow write times to gluster disk

On 05/17/17 02:02, Pranith Kumar Karampuri wrote: On Tue, May 16, 2017 at 9:38 PM, Joe Julian > wrote: On 04/13/17 23:50, Pranith Kumar Karampuri wrote: On Sat, Apr 8, 2017 at 10:28 AM, Ravishankar N

Re: [Gluster-users] Slow write times to gluster disk

On Tue, May 16, 2017 at 9:38 PM, Joe Julian wrote: > On 04/13/17 23:50, Pranith Kumar Karampuri wrote: > > > > On Sat, Apr 8, 2017 at 10:28 AM, Ravishankar N > wrote: > >> Hi Pat, >> >> I'm assuming you are using gluster native (fuse mount). If it

Re: [Gluster-users] Slow write times to gluster disk

On Tue, May 16, 2017 at 9:20 PM, Pat Haley wrote: > > Hi Pranith, > > Sorry for the delay. I never saw received your reply (but I did receive > Ben Turner's follow-up to your reply). So we tried to create a gluster > volume under /home using different variations of > > gluster

Re: [Gluster-users] Slow write times to gluster disk

On 04/13/17 23:50, Pranith Kumar Karampuri wrote: On Sat, Apr 8, 2017 at 10:28 AM, Ravishankar N > wrote: Hi Pat, I'm assuming you are using gluster native (fuse mount). If it helps, you could try mounting it via gluster

Re: [Gluster-users] Slow write times to gluster disk

On 05/10/17 14:18, Pat Haley wrote: Hi Pranith, Since we are mounting the partitions as the bricks, I tried the dd test writing to /.glusterfs/. The results without oflag=sync were 1.6 Gb/s (faster than gluster but not as fast as I was expecting given the 1.2 Gb/s to the no-gluster area w/

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, Sorry for the delay. I never saw received your reply (but I did receive Ben Turner's follow-up to your reply). So we tried to create a gluster volume under /home using different variations of gluster volume create test-volume mseas-data2:/home/gbrick_test_1

Re: [Gluster-users] Slow write times to gluster disk

- Original Message - > From: "Pranith Kumar Karampuri" <pkara...@redhat.com> > To: "Pat Haley" <pha...@mit.edu> > Cc: gluster-users@gluster.org, "Steve Postma" <spos...@ztechnet.com> > Sent: Friday, May 12, 2017 11:17:11 PM >

Re: [Gluster-users] Slow write times to gluster disk

On Sat, May 13, 2017 at 8:44 AM, Pranith Kumar Karampuri < pkara...@redhat.com> wrote: > > > On Fri, May 12, 2017 at 8:04 PM, Pat Haley wrote: > >> >> Hi Pranith, >> >> My question was about setting up a gluster volume on an ext4 partition. >> I thought we had the bricks mounted

Re: [Gluster-users] Slow write times to gluster disk

On Fri, May 12, 2017 at 8:04 PM, Pat Haley wrote: > > Hi Pranith, > > My question was about setting up a gluster volume on an ext4 partition. I > thought we had the bricks mounted as xfs for compatibility with gluster? > Oh that should not be a problem. It works fine. > > Pat

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, My question was about setting up a gluster volume on an ext4 partition. I thought we had the bricks mounted as xfs for compatibility with gluster? Pat On 05/11/2017 12:06 PM, Pranith Kumar Karampuri wrote: On Thu, May 11, 2017 at 9:32 PM, Pat Haley

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, The /home partition is mounted as ext4 /home ext4defaults,usrquota,grpquota 1 2 The brick partitions are mounted ax xfs /mnt/brick1 xfs defaults0 0 /mnt/brick2 xfs defaults0 0 Will this cause a problem with creating a volume under /home?

Re: [Gluster-users] Slow write times to gluster disk

On Thu, May 11, 2017 at 8:57 PM, Pat Haley wrote: > > Hi Pranith, > > Unfortunately, we don't have similar hardware for a small scale test. All > we have is our production hardware. > You said something about /home partition which has lesser disks, we can create plain

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, Unfortunately, we don't have similar hardware for a small scale test. All we have is our production hardware. Pat On 05/11/2017 07:05 AM, Pranith Kumar Karampuri wrote: On Thu, May 11, 2017 at 2:48 AM, Pat Haley > wrote: Hi

Re: [Gluster-users] Slow write times to gluster disk

On Thu, May 11, 2017 at 2:48 AM, Pat Haley wrote: > > Hi Pranith, > > Since we are mounting the partitions as the bricks, I tried the dd test > writing to /.glusterfs/. The > results without oflag=sync were 1.6 Gb/s (faster than gluster but not as > fast as I was expecting given

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, Since we are mounting the partitions as the bricks, I tried the dd test writing to /.glusterfs/. The results without oflag=sync were 1.6 Gb/s (faster than gluster but not as fast as I was expecting given the 1.2 Gb/s to the no-gluster area w/ fewer disks). Pat On 05/10/2017

Re: [Gluster-users] Slow write times to gluster disk

On Wed, May 10, 2017 at 10:15 PM, Pat Haley wrote: > > Hi Pranith, > > Not entirely sure (this isn't my area of expertise). I'll run your answer > by some other people who are more familiar with this. > > I am also uncertain about how to interpret the results when we also add >

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, Not entirely sure (this isn't my area of expertise). I'll run your answer by some other people who are more familiar with this. I am also uncertain about how to interpret the results when we also add the dd tests writing to the /home area (no gluster, still on the same

Re: [Gluster-users] Slow write times to gluster disk

Okay good. At least this validates my doubts. Handling O_SYNC in gluster NFS and fuse is a bit different. When application opens a file with O_SYNC on fuse mount then each write syscall has to be written to disk as part of the syscall where as in case of NFS, there is no concept of open. NFS

Re: [Gluster-users] Slow write times to gluster disk

Without the oflag=sync and only a single test of each, the FUSE is going faster than NFS: FUSE: mseas-data2(dri_nascar)% dd if=/dev/zero count=4096 bs=1048576 of=zeros.txt conv=sync 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 7.46961 s, 575 MB/s NFS

Re: [Gluster-users] Slow write times to gluster disk

Could you let me know the speed without oflag=sync on both the mounts? No need to collect profiles. On Wed, May 10, 2017 at 9:17 PM, Pat Haley wrote: > > Here is what I see now: > > [root@mseas-data2 ~]# gluster volume info > > Volume Name: data-volume > Type: Distribute >

Re: [Gluster-users] Slow write times to gluster disk

Here is what I see now: [root@mseas-data2 ~]# gluster volume info Volume Name: data-volume Type: Distribute Volume ID: c162161e-2a2d-4dac-b015-f31fd89ceb18 Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: mseas-data2:/mnt/brick1 Brick2: mseas-data2:/mnt/brick2 Options

Re: [Gluster-users] Slow write times to gluster disk

Is this the volume info you have? >* [root at mseas-data2 > ~]# gluster volume info *>>* Volume Name: data-volume *>* Type: Distribute *>* Volume ID: c162161e-2a2d-4dac-b015-f31fd89ceb18 *>* Status: Started *>* Number

Re: [Gluster-users] Slow write times to gluster disk

Hi, We finally managed to do the dd tests for an NFS-mounted gluster file system. The profile results during that test are in http://mseas.mit.edu/download/phaley/GlusterUsers/profile_gluster_nfs_test The summary of the dd tests are * writing to gluster disk mounted with fuse: 5 Mb/s *

Re: [Gluster-users] Slow write times to gluster disk

Hi, We redid the dd tests (this time using conv=sync oflag=sync to avoid caching questions). The profile results are in http://mseas.mit.edu/download/phaley/GlusterUsers/profile_gluster_fuse_test On 05/05/2017 12:47 PM, Ravishankar N wrote: On 05/05/2017 08:42 PM, Pat Haley wrote: Hi

Re: [Gluster-users] Slow write times to gluster disk

On 05/05/2017 08:42 PM, Pat Haley wrote: Hi Pranith, I presume you are asking for some version of the profile data that just shows the dd test (or a repeat of the dd test). If yes, how do I extract just that data? Yes, that is what he is asking for. Just clear the existing profile info

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, I presume you are asking for some version of the profile data that just shows the dd test (or a repeat of the dd test). If yes, how do I extract just that data? Thanks Pat On 05/05/2017 10:58 AM, Pranith Kumar Karampuri wrote: hi Pat, Let us concentrate on the

Re: [Gluster-users] Slow write times to gluster disk

hi Pat, Let us concentrate on the performance numbers part for now. We will look at the permissions one after this? As per the profile info, only 2.6% of the work-load is writes. There are too many Lookups. Would it be possible to get the data for just the dd test you were doing earlier?

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith & Ravi, A couple of quick questions We have profile turned on. Are there specific queries we should make that would help debug our configuration? (The default profile info was previously sent in http://lists.gluster.org/pipermail/gluster-users/2017-May/030840.html but I'm not

Re: [Gluster-users] Slow write times to gluster disk

On 04/14/2017 10:27 AM, Ravishankar N wrote: I'm not sure if the version you are running (glusterfs 3.7.11 ) works with NFS-Ganesha as the link seems to suggest version >=3.8 as a per-requisite. Adding Soumya for help. If it is not supported, then you might have to go the plain glusterNFS way.

Re: [Gluster-users] Slow write times to gluster disk

On 04/14/2017 12:20 PM, Pranith Kumar Karampuri wrote: On Sat, Apr 8, 2017 at 10:28 AM, Ravishankar N > wrote: Hi Pat, I'm assuming you are using gluster native (fuse mount). If it helps, you could try mounting it via

Re: [Gluster-users] Slow write times to gluster disk

On Sat, Apr 8, 2017 at 10:28 AM, Ravishankar N wrote: > Hi Pat, > > I'm assuming you are using gluster native (fuse mount). If it helps, you > could try mounting it via gluster NFS (gnfs) and then see if there is an > improvement in speed. Fuse mounts are slower than gnfs

Re: [Gluster-users] Slow write times to gluster disk

I'm not sure if the version you are running (glusterfs 3.7.11 ) works with NFS-Ganesha as the link seems to suggest version >=3.8 as a per-requisite. Adding Soumya for help. If it is not supported, then you might have to go the plain glusterNFS way. Regards, Ravi On 04/14/2017 03:48 AM, Pat

Re: [Gluster-users] Slow write times to gluster disk

Hi Ravi (and list), We are planning on testing the NFS route to see what kind of speed-up we get. A little research led us to the following: https://gluster.readthedocs.io/en/latest/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/ Is this correct path to take to mount 2 xfs

Re: [Gluster-users] Slow write times to gluster disk

On 04/11/2017 12:42 AM, Pat Haley wrote: Hi Ravi, Thanks for the reply. And yes, we are using the gluster native (fuse) mount. Since this is not my area of expertise I have a few questions (mostly clarifications) Is a factor of 20 slow-down typical when compare a fuse-mounted filesytem

Re: [Gluster-users] Slow write times to gluster disk

Hi Ravi, Thanks for the reply. And yes, we are using the gluster native (fuse) mount. Since this is not my area of expertise I have a few questions (mostly clarifications) Is a factor of 20 slow-down typical when compare a fuse-mounted filesytem versus an NFS-mounted filesystem or should

Re: [Gluster-users] Slow write times to gluster disk

Hi Pat, I'm assuming you are using gluster native (fuse mount). If it helps, you could try mounting it via gluster NFS (gnfs) and then see if there is an improvement in speed. Fuse mounts are slower than gnfs mounts but you get the benefit of avoiding a single point of failure. Unlike fuse

[Gluster-users] Slow write times to gluster disk

Hi, We noticed a dramatic slowness when writing to a gluster disk when compared to writing to an NFS disk. Specifically when using dd (data duplicator) to write a 4.3 GB file of zeros: * on NFS disk (/home): 9.5 Gb/s * on gluster disk (/gdata): 508 Mb/s The gluser disk is 2 bricks joined