Re: [Gluster-users] Gluster Volume mounted but not able to show the files from mount point

2016-06-06 Thread ABHISHEK PALIWAL 
i am still facing this issue any suggestion

On Fri, May 27, 2016 at 10:48 AM, ABHISHEK PALIWAL 
wrote:

> any hint from the logs..
>
> On Thu, May 26, 2016 at 11:59 AM, ABHISHEK PALIWAL <
> abhishpali...@gmail.com> wrote:
>
>>
>>
>> On Thu, May 26, 2016 at 11:54 AM, Lindsay Mathieson <
>> lindsay.mathie...@gmail.com> wrote:
>>
>>> On 25 May 2016 at 20:25, ABHISHEK PALIWAL 
>>> wrote:
>>> > [2016-05-24 12:10:20.091267] E [MSGID: 113039]
>>> [posix.c:2570:posix_open]
>>> > 0-c_glusterfs-posix: open on
>>> >
>>> /opt/lvmdir/c2/brick/.glusterfs/fb/14/fb147cca-ec09-4259-9dfe-df883219e6a6,
>>> > flags: 2 [No such file or directory]
>>> > [2016-05-24 12:13:17.305773] E [MSGID: 113039]
>>> [posix.c:2570:posix_open]
>>> > 0-c_glusterfs-posix: open on
>>> >
>>> /opt/lvmdir/c2/brick/.glusterfs/fb/14/fb147cca-ec09-4259-9dfe-df883219e6a6,
>>> > flags: 2 [No such file or directory]
>>>
>>> does /opt/lvmdir/c2/brick contain anything? dies it have a .glusterfd
>>> dir?
>>>
>> Yes .glusterfs directory is present and /opt/lvmdir/c2/brick containing
>> files.
>>
>>>
>>> Could the underlying file system mount for that brick have failed?
>>>
>> mount is successful
>>
>> I have doubt on the following logs
>> [2016-05-24 10:40:34.177887] W [MSGID: 113006] [posix.c:3049:posix_flush]
>> 0-c_glusterfs-posix: pfd is NULL on fd=0x3fff8c003e4c [Operation not
>> permitted]
>> [2016-05-24 10:40:34.178022] E [MSGID: 115065]
>> [server-rpc-fops.c:1354:server_flush_cbk] 0-c_glusterfs-server: 3684: FLUSH
>> -2 (a5a5e596-e786-4f48-8f04-431712d98a6b) ==> (Operation not permitted)
>> [Operation not permitted]
>>
>> Like who will call this posix_flush and in which circumstances?
>>
>> Regards,
>> Abhishek
>>
>>>
>>>
>>> --
>>> Lindsay
>>>
>>
>>
>>
>>
>
>
> --
>
>
>
>
> Regards
> Abhishek Paliwal
>



-- 




Regards
Abhishek Paliwal
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Weekly Community Meeting - 01/June/2016

2016-06-06 Thread Mohammed Rafi K C 
The meeting minutes for this weeks meeting are available at

Minutes:
https://meetbot.fedoraproject.org/gluster-meeting/2016-06-01/weekly_community_meeting_01june2016.2016-06-01-12.00.html
Minutes (text) :
https://meetbot.fedoraproject.org/gluster-meeting/2016-06-01/weekly_community_meeting_01june2016.2016-06-01-12.00.txt
Log:
https://meetbot.fedoraproject.org/gluster-meeting/2016-06-01/weekly_community_meeting_01june2016.2016-06-01-12.00.log.html

Meeting summary
---
* Rollcall  (rafi1, 12:00:58)

* Next weeks meeting host  (rafi1, 12:04:35)

* GlusterFS 4.0  (rafi1, 12:07:30)

* GlusterFS 3.8  (rafi1, 12:11:47)
  * HELP: needed to review the backport patches  (rafi1, 12:13:48)
  * LINK:

http://review.gluster.org/#/q/status:open+project:glusterfs+branch:release-3.8
(rafi1, 12:13:55)
  * requesting to feature owners to give their release notes here in
https://public.pad.fsfe.org/p/glusterfs-3.8-release-notes  (rafi1,
12:16:11)
  * LINK:

https://bugzilla.redhat.com/showdependencytree.cgi?id=glusterfs-3.8.0_resolved=1
(rafi1, 12:19:03)
  * LINK:

https://meetbot.fedoraproject.org/gluster-meeting/2016-05-18/weekly_community_meeting_18may2016.2016-05-18-12.06.log.html
(post-factum, 12:20:01)
  * LINK: https://bugzilla.redhat.com/show_bug.cgi?id=1337130   (rafi1,
12:24:07)

* GlusterFS 3.7  (rafi1, 12:26:13)
  * tentative date for 3.7.12 is on June 9th  (rafi1, 12:30:11)

* GlusterFS 3.6  (rafi1, 12:30:25)
  * LINK:
http://www.gluster.org/pipermail/gluster-devel/2016-May/049677.html
(anoopcs, 12:30:38)
  * targeted release for next 3.6 version ie 3.6.10 will be around 26 of
this month  (rafi1, 12:37:42)

* GlusterFS 3.5  (rafi1, 12:38:04)

* NFS Ganesha  (rafi1, 12:39:32)

* Samba  (rafi1, 12:41:11)

* samba  (rafi1, 12:45:16)

* AI from last week  (rafi1, 12:48:48)

* kshlm/csim/nigelb to set up faux/pseudo user email for gerrit,
  bugzilla, github  (rafi1, 12:49:17)
  * ACTION: kshlm/csim to set up faux/pseudo user email for gerrit,
bugzilla, github  (rafi1, 12:52:27)

* aravinda/amye to check on some blog posts being distorted on
  blog.gluster.org, josferna's post in particular  (rafi1, 12:52:48)
  * ACTION: aravindavk/amye to check on some blog posts being distorted
on blog.gluster.org, josferna's post in particular  (rafi1,
12:54:47)

* hagarth to announce release strategy after getting inputs from amye
  (rafi1, 12:55:19)
  * LINK:
http://www.gluster.org/pipermail/gluster-devel/2016-May/049677.html
(rafi1, 12:55:44)

* kshlm to check with reported of 3.6 leaks on backport need  (rafi1,
  12:56:49)
  * ACTION: kshlm to check with reported of 3.6 leaks on backport need
(rafi1, 12:57:40)

* rastar to look at 3.6 builds failures on BSD  (rafi1, 12:57:51)
  * ACTION: rastar to look at 3.6 builds failures on BSD  (rafi1,
12:58:25)

* Open floor  (rafi1, 12:58:48)

Meeting ended at 13:01:04 UTC.




Action Items

* kshlm/csim to set up faux/pseudo user email for gerrit, bugzilla,
  github
* aravindavk/amye to check on some blog posts being distorted on
  blog.gluster.org, josferna's post in particular
* kshlm to check with reported of 3.6 leaks on backport need
* rastar to look at 3.6 builds failures on BSD




Action Items, by person
---
* **UNASSIGNED**
  * kshlm/csim to set up faux/pseudo user email for gerrit, bugzilla,
github
  * aravindavk/amye to check on some blog posts being distorted on
blog.gluster.org, josferna's post in particular
  * kshlm to check with reported of 3.6 leaks on backport need
  * rastar to look at 3.6 builds failures on BSD




People Present (lines said)
---
* rafi1 (96)
* jiffin (26)
* post-factum (18)
* kkeithley (6)
* anoopcs (4)
* olia (4)
* zodbot (3)
* nigelb (1)
* karthik___ (1)
* glusterbot (1)
* overclk (1)
* partner (1)
* rjoseph (1)





___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] snapshot removal failed on one node how to recover (3.7.11)

2016-06-06 Thread Alastair Neil 
No one has any suggestions?  Would this scenario I have been toying with
work:  remove the brick from the node with the out of sync snapshots,
destroy all associated logical volumes, and  then add the brick back as an
arbiter node?


On 1 June 2016 at 13:40, Alastair Neil  wrote:

> I have a replica 3 volume that has snapshot scheduled using
> snap_scheduler.py
>
> I recently tried to remove a snapshot and the command failed on one node:
>
> snapshot delete: failed: Commit failed on gluster0.vsnet.gmu.edu. Please
>> check log file for details.
>> Snapshot command failed
>
>
> How do I recover from this failure.  Clearly I need to remove the snapshot
> from the offending server but this does not seem possible as the snapshot
> no longer exists on the other two nodes.
> Suggestions welcome.
>
> -Alastair
>
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] files on glusterfs disappears

2016-06-06 Thread Jeff Darcy 
> This could be because of nufa xlator. As you say the files are present on the
> brick I don't suspect RDMA here.

Agreed.

> Is nufa still supported? Could this a bug in nufa + dht?

Until we explicitly decide to stop building and distributing it, it's still
"supported" in some sense, but only to the extent that there's someone
available to look at it.  Unfortunately, nobody has that as an assignment
and our tests for NUFA are minimal so they're not likely to detect breakage
automatically.  With the amount of change we've seen to DHT over the last
several months, it's entirely possible that a NUFA bug or two has crept in.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] files on glusterfs disappears

2016-06-06 Thread Raghavendra Talur 
This could be because of nufa xlator. As you say the files are present on
the brick I don't suspect RDMA here.

Raghavendra,

Is nufa still supported? Could this a bug in nufa + dht?
On 06-Jun-2016 10:29 PM, "Fedele Stabile" 
wrote:

> I add some information about my cluster:
> [root@wn001 glusterfs]# gluster volume info
>
> Volume Name: scratch
> Type: Distribute
> Volume ID: fc6f18b6-a06c-4fdf-ac08-23e9b4f8053e
> Status: Started
> Number of Bricks: 32
> Transport-type: rdma
> Bricks:
> Brick1: ib-wn001:/bricks/brick1/gscratch0
> Brick2: ib-wn002:/bricks/brick1/gscratch0
> 
> Brick31: ib-wn032:/bricks/brick1/gscratch0
> Options Reconfigured:
> features.scrub-freq: daily
> features.scrub: Active
> features.bitrot: on
> cluster.nufa: on
> performance.readdir-ahead: on
> config.transport: rdma
> nfs.disable: true
> [root@wn001 glusterfs]#
> [root@wn001 glusterfs]# gluster volume status
> Status of volume: scratch
> Gluster process TCP Port  RDMA
> Port  Online  Pid
> -
> -
> Brick ib-
> wn001:/bricks/brick1/gscratch0 0 49155  Y   3380
> Brick ib-
> wn002:/bricks/brick1/gscratch0 0 49155  Y   3463
> 
> Brick ib-
> wn032:/bricks/brick1/gscratch0 0 49152  Y   3496
> Bitrot Daemon on ib-wn0001
> N/A   N/AY   9150
> Scrubber Daemon on ib-wn001
> N/A   N/AY   9159
> .
> Bitrot Daemon on ib-
> wn032   N/A   N/AY   31107
> Scrubber Daemon on ib-
> wn032 N/A   N/AY   31114
>
> Task Status of Volume scratch
> -
> -
> Task : Rebalance
> ID   : 38373576-dcb5-469f-9ae1-42c56ff445e5
> Status   : completed
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] files on glusterfs disappears

2016-06-06 Thread Fedele Stabile 
I add some information about my cluster:
[root@wn001 glusterfs]# gluster volume info
 
Volume Name: scratch
Type: Distribute
Volume ID: fc6f18b6-a06c-4fdf-ac08-23e9b4f8053e
Status: Started
Number of Bricks: 32
Transport-type: rdma
Bricks:
Brick1: ib-wn001:/bricks/brick1/gscratch0
Brick2: ib-wn002:/bricks/brick1/gscratch0

Brick31: ib-wn032:/bricks/brick1/gscratch0
Options Reconfigured:
features.scrub-freq: daily
features.scrub: Active
features.bitrot: on
cluster.nufa: on
performance.readdir-ahead: on
config.transport: rdma
nfs.disable: true
[root@wn001 glusterfs]# 
[root@wn001 glusterfs]# gluster volume status
Status of volume: scratch
Gluster process TCP Port  RDMA
Port  Online  Pid
-
-
Brick ib-
wn001:/bricks/brick1/gscratch0 0 49155  Y   3380 
Brick ib-
wn002:/bricks/brick1/gscratch0 0 49155  Y   3463 

Brick ib-
wn032:/bricks/brick1/gscratch0 0 49152  Y   3496 
Bitrot Daemon on ib-wn0001                  
N/A   N/AY   9150 
Scrubber Daemon on ib-wn001                
N/A   N/AY   9159 
.
Bitrot Daemon on ib-
wn032   N/A   N/AY   31107
Scrubber Daemon on ib-
wn032 N/A   N/AY   31114
 
Task Status of Volume scratch
-
-
Task : Rebalance   
ID   : 38373576-dcb5-469f-9ae1-42c56ff445e5
Status   : completed   
 
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] files on glusterfs disappears

2016-06-06 Thread Fedele Stabile 
Good morning to all the community.
I have a problem and I would like to ask for your kind help. 
It happens that newly written files from an MPI-application that uses
InfiniBand disappear from glusterfs but they are present in a brick... 
Please, can anyone help me to solve this problem?

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] implementation of RPC in glusterFS

2016-06-06 Thread 袁仲 
thanks for your reply

On Sun, Jun 5, 2016 at 10:42 PM, Kaleb Keithley  wrote:

>
>
> - Original Message -
> > From: "袁仲" >
> >
> >
> > I have check the source code of glusterFS, the communication between cli
> and
> > glustered , glustered and glusterd is depend on RPC. But it is different
> > from the RPC program i wrote myself that I use rpcgen to create
> client-stub
> > and server-stub, but glusterFS does not. so, my question is,
> >
> > Does glusterfs have implement RPC by itself, and if does, what’s the
> > difference from the RPC program that use rpcgen.
>
> GlusterFS uses rpcgen too.
>
> The protocol is defined by the XDR files in .../rpc/xdr/src/*.x.
>
> The stubs are generated using rpcgen. See .../build-aux/xdrgen
>
> --
>
> Kaleb
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] [Gluster-devel] Huge VSZ (VIRT) usage by glustershd on dummy node

2016-06-06 Thread Oleksandr Natalenko 

Also, I see lots of entries in pmap output:

===
7ef9ff8f3000  4K -   [ anon ]
7ef9ff8f4000   8192K rw---   [ anon ]
7efa000f4000  4K -   [ anon ]
7efa000f5000   8192K rw---   [ anon ]
===

If I sum them, I get the following:

===
# pmap 15109 | grep '[ anon ]' | grep 8192K | wc -l
9261
$ echo "9261*(8192+4)" | bc
75903156
===

Which is something like 70G+ I have got in VIRT.

06.06.2016 11:24, Oleksandr Natalenko написав:

Hello.

We use v3.7.11, replica 2 setup between 2 nodes + 1 dummy node for
keeping volumes metadata.

Now we observe huge VSZ (VIRT) usage by glustershd on dummy node:

===
root 15109  0.0 13.7 76552820 535272 ? Ssl  тра26   2:11
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid -l
/var/log/glusterfs/glustershd.log -S
/var/run/gluster/7848e17764dd4ba80f4623aecb91b07a.socket
--xlator-option
*replicate*.node-uuid=80bc95e1-2027-4a96-bb66-d9c8ade624d7
===

that is ~73G. RSS seems to be OK (~522M). Here is the statedump of
glustershd process: [1]

Also, here is sum of sizes, presented in statedump:

===
# cat /var/run/gluster/glusterdump.15109.dump.1465200139 | awk -F '='
'BEGIN {sum=0} /^size=/ {sum+=$2} END {print sum}'
353276406
===

That is ~337 MiB.

Also, here are VIRT values from 2 replica nodes:

===
root 24659  0.0  0.3 5645836 451796 ?  Ssl  тра24   3:28
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid -l
/var/log/glusterfs/glustershd.log -S
/var/run/gluster/44ec3f29003eccedf894865107d5db90.socket
--xlator-option
*replicate*.node-uuid=a19afcc2-e26c-43ce-bca6-d27dc1713e87
root 18312  0.0  0.3 6137500 477472 ?  Ssl  тра19   6:37
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid -l
/var/log/glusterfs/glustershd.log -S
/var/run/gluster/1670a3abbd1eea968126eb6f5be20322.socket
--xlator-option
*replicate*.node-uuid=52dca21b-c81c-48b5-9de2-1ed37987fbc2
===

Those are 5 to 6G, which is much less than dummy node has, but still
look too big for us.

Should we care about huge VIRT value on dummy node? Also, how one
would debug that?

Regards,
  Oleksandr.

[1] https://gist.github.com/d2cfa25251136512580220fcdb8a6ce6
___
Gluster-devel mailing list
gluster-de...@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] [Gluster-devel] Huge VSZ (VIRT) usage by glustershd on dummy node

2016-06-06 Thread Oleksandr Natalenko 
I believe, multi-threaded shd has not been merged at least into 3.7 
branch prior to 3.7.11 (incl.), because I've found this [1].


[1] https://www.gluster.org/pipermail/maintainers/2016-April/000628.html

06.06.2016 12:21, Kaushal M написав:

Has multi-threaded SHD been merged into 3.7.* by any chance? If not,
what I'm saying below doesn't apply.

We saw problems when encrypted transports were used, because the RPC
layer was not reaping threads (doing pthread_join) when a connection
ended. This lead to similar observations of huge VIRT and relatively
small RSS.

I'm not sure how multi-threaded shd works, but it could be leaking
threads in a similar way.

On Mon, Jun 6, 2016 at 1:54 PM, Oleksandr Natalenko
 wrote:

Hello.

We use v3.7.11, replica 2 setup between 2 nodes + 1 dummy node for 
keeping

volumes metadata.

Now we observe huge VSZ (VIRT) usage by glustershd on dummy node:

===
root 15109  0.0 13.7 76552820 535272 ? Ssl  тра26   2:11
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid -l
/var/log/glusterfs/glustershd.log -S
/var/run/gluster/7848e17764dd4ba80f4623aecb91b07a.socket 
--xlator-option

*replicate*.node-uuid=80bc95e1-2027-4a96-bb66-d9c8ade624d7
===

that is ~73G. RSS seems to be OK (~522M). Here is the statedump of
glustershd process: [1]

Also, here is sum of sizes, presented in statedump:

===
# cat /var/run/gluster/glusterdump.15109.dump.1465200139 | awk -F '=' 
'BEGIN

{sum=0} /^size=/ {sum+=$2} END {print sum}'
353276406
===

That is ~337 MiB.

Also, here are VIRT values from 2 replica nodes:

===
root 24659  0.0  0.3 5645836 451796 ?  Ssl  тра24   3:28
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid -l
/var/log/glusterfs/glustershd.log -S
/var/run/gluster/44ec3f29003eccedf894865107d5db90.socket 
--xlator-option

*replicate*.node-uuid=a19afcc2-e26c-43ce-bca6-d27dc1713e87
root 18312  0.0  0.3 6137500 477472 ?  Ssl  тра19   6:37
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid -l
/var/log/glusterfs/glustershd.log -S
/var/run/gluster/1670a3abbd1eea968126eb6f5be20322.socket 
--xlator-option

*replicate*.node-uuid=52dca21b-c81c-48b5-9de2-1ed37987fbc2
===

Those are 5 to 6G, which is much less than dummy node has, but still 
look

too big for us.

Should we care about huge VIRT value on dummy node? Also, how one 
would

debug that?

Regards,
  Oleksandr.

[1] https://gist.github.com/d2cfa25251136512580220fcdb8a6ce6
___
Gluster-devel mailing list
gluster-de...@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] [Gluster-devel] Huge VSZ (VIRT) usage by glustershd on dummy node

2016-06-06 Thread Kaushal M 
Has multi-threaded SHD been merged into 3.7.* by any chance? If not,
what I'm saying below doesn't apply.

We saw problems when encrypted transports were used, because the RPC
layer was not reaping threads (doing pthread_join) when a connection
ended. This lead to similar observations of huge VIRT and relatively
small RSS.

I'm not sure how multi-threaded shd works, but it could be leaking
threads in a similar way.

On Mon, Jun 6, 2016 at 1:54 PM, Oleksandr Natalenko
 wrote:
> Hello.
>
> We use v3.7.11, replica 2 setup between 2 nodes + 1 dummy node for keeping
> volumes metadata.
>
> Now we observe huge VSZ (VIRT) usage by glustershd on dummy node:
>
> ===
> root 15109  0.0 13.7 76552820 535272 ? Ssl  тра26   2:11
> /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
> /var/lib/glusterd/glustershd/run/glustershd.pid -l
> /var/log/glusterfs/glustershd.log -S
> /var/run/gluster/7848e17764dd4ba80f4623aecb91b07a.socket --xlator-option
> *replicate*.node-uuid=80bc95e1-2027-4a96-bb66-d9c8ade624d7
> ===
>
> that is ~73G. RSS seems to be OK (~522M). Here is the statedump of
> glustershd process: [1]
>
> Also, here is sum of sizes, presented in statedump:
>
> ===
> # cat /var/run/gluster/glusterdump.15109.dump.1465200139 | awk -F '=' 'BEGIN
> {sum=0} /^size=/ {sum+=$2} END {print sum}'
> 353276406
> ===
>
> That is ~337 MiB.
>
> Also, here are VIRT values from 2 replica nodes:
>
> ===
> root 24659  0.0  0.3 5645836 451796 ?  Ssl  тра24   3:28
> /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
> /var/lib/glusterd/glustershd/run/glustershd.pid -l
> /var/log/glusterfs/glustershd.log -S
> /var/run/gluster/44ec3f29003eccedf894865107d5db90.socket --xlator-option
> *replicate*.node-uuid=a19afcc2-e26c-43ce-bca6-d27dc1713e87
> root 18312  0.0  0.3 6137500 477472 ?  Ssl  тра19   6:37
> /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
> /var/lib/glusterd/glustershd/run/glustershd.pid -l
> /var/log/glusterfs/glustershd.log -S
> /var/run/gluster/1670a3abbd1eea968126eb6f5be20322.socket --xlator-option
> *replicate*.node-uuid=52dca21b-c81c-48b5-9de2-1ed37987fbc2
> ===
>
> Those are 5 to 6G, which is much less than dummy node has, but still look
> too big for us.
>
> Should we care about huge VIRT value on dummy node? Also, how one would
> debug that?
>
> Regards,
>   Oleksandr.
>
> [1] https://gist.github.com/d2cfa25251136512580220fcdb8a6ce6
> ___
> Gluster-devel mailing list
> gluster-de...@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Huge VSZ (VIRT) usage by glustershd on dummy node

2016-06-06 Thread Oleksandr Natalenko 

Hello.

We use v3.7.11, replica 2 setup between 2 nodes + 1 dummy node for 
keeping volumes metadata.


Now we observe huge VSZ (VIRT) usage by glustershd on dummy node:

===
root 15109  0.0 13.7 76552820 535272 ? Ssl  тра26   2:11 
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p 
/var/lib/glusterd/glustershd/run/glustershd.pid -l 
/var/log/glusterfs/glustershd.log -S 
/var/run/gluster/7848e17764dd4ba80f4623aecb91b07a.socket --xlator-option 
*replicate*.node-uuid=80bc95e1-2027-4a96-bb66-d9c8ade624d7

===

that is ~73G. RSS seems to be OK (~522M). Here is the statedump of 
glustershd process: [1]


Also, here is sum of sizes, presented in statedump:

===
# cat /var/run/gluster/glusterdump.15109.dump.1465200139 | awk -F '=' 
'BEGIN {sum=0} /^size=/ {sum+=$2} END {print sum}'

353276406
===

That is ~337 MiB.

Also, here are VIRT values from 2 replica nodes:

===
root 24659  0.0  0.3 5645836 451796 ?  Ssl  тра24   3:28 
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p 
/var/lib/glusterd/glustershd/run/glustershd.pid -l 
/var/log/glusterfs/glustershd.log -S 
/var/run/gluster/44ec3f29003eccedf894865107d5db90.socket --xlator-option 
*replicate*.node-uuid=a19afcc2-e26c-43ce-bca6-d27dc1713e87
root 18312  0.0  0.3 6137500 477472 ?  Ssl  тра19   6:37 
/usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p 
/var/lib/glusterd/glustershd/run/glustershd.pid -l 
/var/log/glusterfs/glustershd.log -S 
/var/run/gluster/1670a3abbd1eea968126eb6f5be20322.socket --xlator-option 
*replicate*.node-uuid=52dca21b-c81c-48b5-9de2-1ed37987fbc2

===

Those are 5 to 6G, which is much less than dummy node has, but still 
look too big for us.


Should we care about huge VIRT value on dummy node? Also, how one would 
debug that?


Regards,
  Oleksandr.

[1] https://gist.github.com/d2cfa25251136512580220fcdb8a6ce6
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users