Re: [Gluster-devel] [Gluster-users] v3.6.2
Your nfs.log file has the following lines, ``` [2015-01-26 20:06:58.298078] E [rpcsvc.c:1303:rpcsvc_program_register_portmap] 0-rpc-service: Could not register with portmap 100021 4 38468 [2015-01-26 20:06:58.298108] E [nfs.c:331:nfs_init_versions] 0-nfs: Program NLM4 registration failed ``` The Gluster NFS server has it's own NLM (nlockmgr) implementation. You said that you have the nfslock service on. Can you turn that off and try again? ~kaushal On Tue, Jan 27, 2015 at 11:21 AM, David F. Robinson < david.robin...@corvidtec.com> wrote: > On a different system where gluster-NFS (not kernel-nfs) is working > properly shows the following: > > [root@gfs01a glusterfs]# rpcinfo -p >program vers proto port service > 104 tcp111 portmapper > 103 tcp111 portmapper > 102 tcp111 portmapper > 104 udp111 portmapper > 103 udp111 portmapper > 102 udp111 portmapper > 153 tcp 38465 mountd > 151 tcp 38466 mountd > 133 tcp 2049 nfs > 1000241 udp 42413 status > 1000241 tcp 35424 status > 1000214 tcp 38468 nlockmgr > 1000211 udp801 nlockmgr > 1002273 tcp 2049 nfs_acl > 1000211 tcp804 nlockmgr > > [root@gfs01a glusterfs]# /etc/init.d/nfs status > rpc.svcgssd is stopped > rpc.mountd is stopped > nfsd is stopped > rpc.rquotad is stopped > > > > -- Original Message -- > From: "Joe Julian" > To: "Kaushal M" ; "David F. Robinson" < > david.robin...@corvidtec.com> > Cc: "Gluster Users" ; "Gluster Devel" < > gluster-devel@gluster.org> > Sent: 1/27/2015 12:48:49 AM > Subject: Re: [Gluster-devel] [Gluster-users] v3.6.2 > > > If that was true, wouldn't it not "Connection refused" because the kernel > nfs is listening? > > On January 26, 2015 9:43:34 PM PST, Kaushal M > wrote: >> >> Seems like you have the kernel NFS server running. The `rpcinfo -p` >> output you provided shows that there are other mountd, nfs and nlockmgr >> services running your system. Gluster NFS server requires that the kernel >> nfs services be disabled and not running. >> >> ~kaushal >> >> On Tue, Jan 27, 2015 at 10:56 AM, David F. Robinson < >> david.robin...@corvidtec.com> wrote: >> >>> [root@gfs01bkp ~]# gluster volume status homegfs_bkp >>> Status of volume: homegfs_bkp >>> Gluster process PortOnline >>> Pid >>> >>> -- >>> Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs >>> _bkp49152 Y >>> 4087 >>> Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs >>> _bkp49155 Y >>> 4092 >>> NFS Server on localhost N/A N >>> N/A >>> >>> Task Status of Volume homegfs_bkp >>> >>> -- >>> Task : Rebalance >>> ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 >>> Status : completed >>> >>> >>> >>> >>> >>> -- Original Message -- >>> From: "Atin Mukherjee" >>> To: "Pranith Kumar Karampuri" ; "Justin Clift" < >>> jus...@gluster.org>; "David F. Robinson" >>> Cc: "Gluster Users" ; "Gluster Devel" < >>> gluster-devel@gluster.org> >>> Sent: 1/26/2015 11:51:13 PM >>> Subject: Re: [Gluster-devel] v3.6.2 >>> >>> On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: > > On 01/26/2015 09:41 PM, Justin Clift wrote: > >> On 26 Jan 2015, at 14:50, David F. Robinson >> wrote: >> >>> I have a server with v3.6.2 from which I cannot mount using NFS. The >>> FUSE mount works, however, I cannot get the NFS mount to work. From >>> /var/log/message: >>>Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server >>
Re: [Gluster-devel] [Gluster-users] v3.6.2
I think you get a connection refused if there are no servers running as well. On Tue, Jan 27, 2015 at 11:18 AM, Joe Julian wrote: > If that was true, wouldn't it not "Connection refused" because the kernel > nfs is listening? > > On January 26, 2015 9:43:34 PM PST, Kaushal M wrote: >> >> Seems like you have the kernel NFS server running. The `rpcinfo -p` >> output you provided shows that there are other mountd, nfs and nlockmgr >> services running your system. Gluster NFS server requires that the kernel >> nfs services be disabled and not running. >> >> ~kaushal >> >> On Tue, Jan 27, 2015 at 10:56 AM, David F. Robinson < >> david.robin...@corvidtec.com> wrote: >> >>> [root@gfs01bkp ~]# gluster volume status homegfs_bkp >>> Status of volume: homegfs_bkp >>> Gluster process PortOnline >>> Pid >>> >>> -- >>> Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs >>> _bkp49152 Y >>> 4087 >>> Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs >>> _bkp49155 Y >>> 4092 >>> NFS Server on localhost N/A N >>> N/A >>> >>> Task Status of Volume homegfs_bkp >>> >>> -- >>> Task : Rebalance >>> ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 >>> Status : completed >>> >>> >>> >>> >>> >>> -- Original Message -- >>> From: "Atin Mukherjee" >>> To: "Pranith Kumar Karampuri" ; "Justin Clift" < >>> jus...@gluster.org>; "David F. Robinson" >>> Cc: "Gluster Users" ; "Gluster Devel" < >>> gluster-devel@gluster.org> >>> Sent: 1/26/2015 11:51:13 PM >>> Subject: Re: [Gluster-devel] v3.6.2 >>> >>> On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: > > On 01/26/2015 09:41 PM, Justin Clift wrote: > >> On 26 Jan 2015, at 14:50, David F. Robinson >> wrote: >> >>> I have a server with v3.6.2 from which I cannot mount using NFS. The >>> FUSE mount works, however, I cannot get the NFS mount to work. From >>> /var/log/message: >>>Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> I also am continually getting the following errors in >>> /var/log/glusterfs: >>>[root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log >>> [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (I
Re: [Gluster-devel] [Gluster-users] v3.6.2
On a different system where gluster-NFS (not kernel-nfs) is working properly shows the following: [root@gfs01a glusterfs]# rpcinfo -p program vers proto port service 104 tcp111 portmapper 103 tcp111 portmapper 102 tcp111 portmapper 104 udp111 portmapper 103 udp111 portmapper 102 udp111 portmapper 153 tcp 38465 mountd 151 tcp 38466 mountd 133 tcp 2049 nfs 1000241 udp 42413 status 1000241 tcp 35424 status 1000214 tcp 38468 nlockmgr 1000211 udp801 nlockmgr 1002273 tcp 2049 nfs_acl 1000211 tcp804 nlockmgr [root@gfs01a glusterfs]# /etc/init.d/nfs status rpc.svcgssd is stopped rpc.mountd is stopped nfsd is stopped rpc.rquotad is stopped -- Original Message -- From: "Joe Julian" To: "Kaushal M" ; "David F. Robinson" Cc: "Gluster Users" ; "Gluster Devel" Sent: 1/27/2015 12:48:49 AM Subject: Re: [Gluster-devel] [Gluster-users] v3.6.2 If that was true, wouldn't it not "Connection refused" because the kernel nfs is listening? On January 26, 2015 9:43:34 PM PST, Kaushal M wrote: Seems like you have the kernel NFS server running. The `rpcinfo -p` output you provided shows that there are other mountd, nfs and nlockmgr services running your system. Gluster NFS server requires that the kernel nfs services be disabled and not running. ~kaushal On Tue, Jan 27, 2015 at 10:56 AM, David F. Robinson wrote: [root@gfs01bkp ~]# gluster volume status homegfs_bkp Status of volume: homegfs_bkp Gluster process Port Online Pid -- Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs _bkp49152 Y 4087 Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs _bkp49155 Y 4092 NFS Server on localhost N/A N N/A Task Status of Volume homegfs_bkp -- Task : Rebalance ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 Status : completed -- Original Message -- From: "Atin Mukherjee" To: "Pranith Kumar Karampuri" ; "Justin Clift" ; "David F. Robinson" Cc: "Gluster Users" ; "Gluster Devel" Sent: 1/26/2015 11:51:13 PM Subject: Re: [Gluster-devel] v3.6.2 On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: On 01/26/2015 09:41 PM, Justin Clift wrote: On 26 Jan 2015, at 14:50, David F. Robinson wrote: I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Inval
Re: [Gluster-devel] [Gluster-users] v3.6.2
If that was true, wouldn't it not "Connection refused" because the kernel nfs is listening? On January 26, 2015 9:43:34 PM PST, Kaushal M wrote: >Seems like you have the kernel NFS server running. The `rpcinfo -p` >output >you provided shows that there are other mountd, nfs and nlockmgr >services >running your system. Gluster NFS server requires that the kernel nfs >services be disabled and not running. > >~kaushal > >On Tue, Jan 27, 2015 at 10:56 AM, David F. Robinson < >david.robin...@corvidtec.com> wrote: > >> [root@gfs01bkp ~]# gluster volume status homegfs_bkp >> Status of volume: homegfs_bkp >> Gluster process Port >Online Pid >> >> -- >> Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs >> _bkp49152 Y >> 4087 >> Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs >> _bkp49155 Y >> 4092 >> NFS Server on localhost N/A N > N/A >> >> Task Status of Volume homegfs_bkp >> >> -- >> Task : Rebalance >> ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 >> Status : completed >> >> >> >> >> >> -- Original Message -- >> From: "Atin Mukherjee" >> To: "Pranith Kumar Karampuri" ; "Justin Clift" < >> jus...@gluster.org>; "David F. Robinson" > >> Cc: "Gluster Users" ; "Gluster Devel" < >> gluster-devel@gluster.org> >> Sent: 1/26/2015 11:51:13 PM >> Subject: Re: [Gluster-devel] v3.6.2 >> >> >>> >>> On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: >>> On 01/26/2015 09:41 PM, Justin Clift wrote: > On 26 Jan 2015, at 14:50, David F. Robinson > wrote: > >> I have a server with v3.6.2 from which I cannot mount using NFS. >The >> FUSE mount works, however, I cannot get the NFS mount to work. >From >> /var/log/message: >>Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server >> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >> I also am continually getting the following errors in >> /var/log/glusterfs: >>[root@gfs01bkp glusterfs]# tail -f >etc-glusterfs-glusterd.vol.log >> [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] >> 0-management: readv on >> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >> argument) >> [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] >> 0-management: readv on >> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >> argument) >> [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] >> 0-management: readv on >> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >> argument) >> [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] >> 0-management: readv on >> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >> argument) >> [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] >> 0-management: readv on >> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >> argument) >> [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] >> 0-management: readv on >> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >> argument) >> [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] >> 0-management: readv on >> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >> argument) >> [2015-01-26 14:42:12.2
Re: [Gluster-devel] [Gluster-users] v3.6.2
chkconfig shows nfslock setup to run but not nfs. I confirmed this by doing a /etc/init.d/nfs status. That service is not running. Nothing has changed in the setup and NFS is working for my other gluster systems. However, I had to power down this particular system to do some A/C upgrades in our datacenter. When I powered it back up, NFS through gluster will no longer start. David -- Original Message -- From: "Kaushal M" To: "David F. Robinson" Cc: "Atin Mukherjee" ; "Pranith Kumar Karampuri" ; "Justin Clift" ; "Gluster Users" ; "Gluster Devel" Sent: 1/27/2015 12:43:34 AM Subject: Re: [Gluster-users] [Gluster-devel] v3.6.2 Seems like you have the kernel NFS server running. The `rpcinfo -p` output you provided shows that there are other mountd, nfs and nlockmgr services running your system. Gluster NFS server requires that the kernel nfs services be disabled and not running. ~kaushal On Tue, Jan 27, 2015 at 10:56 AM, David F. Robinson wrote: [root@gfs01bkp ~]# gluster volume status homegfs_bkp Status of volume: homegfs_bkp Gluster process PortOnline Pid -- Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs _bkp49152 Y 4087 Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs _bkp49155 Y 4092 NFS Server on localhost N/A N N/A Task Status of Volume homegfs_bkp -- Task : Rebalance ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 Status : completed -- Original Message -- From: "Atin Mukherjee" To: "Pranith Kumar Karampuri" ; "Justin Clift" ; "David F. Robinson" Cc: "Gluster Users" ; "Gluster Devel" Sent: 1/26/2015 11:51:13 PM Subject: Re: [Gluster-devel] v3.6.2 On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: On 01/26/2015 09:41 PM, Justin Clift wrote: On 26 Jan 2015, at 14:50, David F. Robinson wrote: I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a
Re: [Gluster-devel] v3.6.2
nfs.log attached. Where is glusterd.log? David -- Original Message -- From: "Atin Mukherjee" To: "David F. Robinson" ; "Pranith Kumar Karampuri" ; "Justin Clift" Cc: "Gluster Users" ; "Gluster Devel" Sent: 1/27/2015 12:42:44 AM Subject: Re: [Gluster-devel] v3.6.2 On 01/27/2015 10:56 AM, David F. Robinson wrote: [root@gfs01bkp ~]# gluster volume status homegfs_bkp Status of volume: homegfs_bkp Gluster process Port Online Pid -- Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs _bkp 49152 Y 4087 Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs _bkp 49155 Y 4092 NFS Server on localhost N/A N N/A Ahh!! here is the problem, your NFS server is not started because of which NFS mount is failing. Can you please share the nfs.log & glusterd.log files. ~Atin Task Status of Volume homegfs_bkp -- Task : Rebalance ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 Status : completed -- Original Message -- From: "Atin Mukherjee" To: "Pranith Kumar Karampuri" ; "Justin Clift" ; "David F. Robinson" Cc: "Gluster Users" ; "Gluster Devel" Sent: 1/26/2015 11:51:13 PM Subject: Re: [Gluster-devel] v3.6.2 On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: On 01/26/2015 09:41 PM, Justin Clift wrote: On 26 Jan 2015, at 14:50, David F. Robinson wrote: I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d07
Re: [Gluster-devel] [Gluster-users] v3.6.2
Seems like you have the kernel NFS server running. The `rpcinfo -p` output you provided shows that there are other mountd, nfs and nlockmgr services running your system. Gluster NFS server requires that the kernel nfs services be disabled and not running. ~kaushal On Tue, Jan 27, 2015 at 10:56 AM, David F. Robinson < david.robin...@corvidtec.com> wrote: > [root@gfs01bkp ~]# gluster volume status homegfs_bkp > Status of volume: homegfs_bkp > Gluster process PortOnline Pid > > -- > Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs > _bkp49152 Y > 4087 > Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs > _bkp49155 Y > 4092 > NFS Server on localhost N/A N N/A > > Task Status of Volume homegfs_bkp > > -- > Task : Rebalance > ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 > Status : completed > > > > > > -- Original Message -- > From: "Atin Mukherjee" > To: "Pranith Kumar Karampuri" ; "Justin Clift" < > jus...@gluster.org>; "David F. Robinson" > Cc: "Gluster Users" ; "Gluster Devel" < > gluster-devel@gluster.org> > Sent: 1/26/2015 11:51:13 PM > Subject: Re: [Gluster-devel] v3.6.2 > > >> >> On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: >> >>> >>> On 01/26/2015 09:41 PM, Justin Clift wrote: >>> On 26 Jan 2015, at 14:50, David F. Robinson wrote: > I have a server with v3.6.2 from which I cannot mount using NFS. The > FUSE mount works, however, I cannot get the NFS mount to work. From > /var/log/message: >Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > I also am continually getting the following errors in > /var/log/glusterfs: >[root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log > [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/r
Re: [Gluster-devel] v3.6.2
On 01/27/2015 10:56 AM, David F. Robinson wrote: > [root@gfs01bkp ~]# gluster volume status homegfs_bkp > Status of volume: homegfs_bkp > Gluster process PortOnline Pid > -- > > Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs > _bkp49152 Y > 4087 > Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs > _bkp49155 Y > 4092 > NFS Server on localhost N/A N N/A Ahh!! here is the problem, your NFS server is not started because of which NFS mount is failing. Can you please share the nfs.log & glusterd.log files. ~Atin > > Task Status of Volume homegfs_bkp > -- > > Task : Rebalance > ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 > Status : completed > > > > > -- Original Message -- > From: "Atin Mukherjee" > To: "Pranith Kumar Karampuri" ; "Justin Clift" > ; "David F. Robinson" > Cc: "Gluster Users" ; "Gluster Devel" > > Sent: 1/26/2015 11:51:13 PM > Subject: Re: [Gluster-devel] v3.6.2 > >> >> >> On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: >>> >>> On 01/26/2015 09:41 PM, Justin Clift wrote: On 26 Jan 2015, at 14:50, David F. Robinson wrote: > I have a server with v3.6.2 from which I cannot mount using NFS. The > FUSE mount works, however, I cannot get the NFS mount to work. From > /var/log/message: >Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > I also am continually getting the following errors in > /var/log/glusterfs: >[root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log > [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] > 0-management: readv on > /var/run/1f0cee5
Re: [Gluster-devel] v3.6.2
[root@gfs01bkp ~]# gluster volume status homegfs_bkp Status of volume: homegfs_bkp Gluster process PortOnline Pid -- Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs _bkp49152 Y 4087 Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs _bkp49155 Y 4092 NFS Server on localhost N/A N N/A Task Status of Volume homegfs_bkp -- Task : Rebalance ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 Status : completed -- Original Message -- From: "Atin Mukherjee" To: "Pranith Kumar Karampuri" ; "Justin Clift" ; "David F. Robinson" Cc: "Gluster Users" ; "Gluster Devel" Sent: 1/26/2015 11:51:13 PM Subject: Re: [Gluster-devel] v3.6.2 On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: On 01/26/2015 09:41 PM, Justin Clift wrote: On 26 Jan 2015, at 14:50, David F. Robinson wrote: I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) I believe this error message comes when the socket file is not present. I see the following commit which changed the location of the sockets. May be Atin may know more. about this: +Atin. Can we get the output of 'gluster volume status' for the volume which you are trying to mount? ~
Re: [Gluster-devel] v3.6.2
On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: > > On 01/26/2015 09:41 PM, Justin Clift wrote: >> On 26 Jan 2015, at 14:50, David F. Robinson >> wrote: >>> I have a server with v3.6.2 from which I cannot mount using NFS. The >>> FUSE mount works, however, I cannot get the NFS mount to work. From >>> /var/log/message: >>> Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server >>> 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>> I also am continually getting the following errors in >>> /var/log/glusterfs: >>> [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log >>> [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) >>> [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] >>> 0-management: readv on >>> /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid >>> argument) > I believe this error message comes when the socket file is not present. > I see the following commit which changed the location of the sockets. > May be Atin may know more. about this: +Atin. > Can we get the output of 'gluster volume status' for the volume which you are trying to mount? ~Atin > Pranith >>> ^C >>> Also, when I try to NFS mount my gluster volume, I am getting >> Any chance there's a network or host based firewall stopping some of >> the ports? >> >> + Justin >> >> -- >> GlusterFS - http://www.gluster.org >> >> An open source, distributed file system scaling to several >> petabytes, and handling thousands of clients. >> >> My personal twitter: twitter.com/realjustinclift >> >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] GlusterFS 4.0 updates needed
I have updated thousand-node-glusterd. http://www.gluster.org/community/documentation/index.php/Features/thousand-node-glusterd#Status ~kp - Original Message - > Updated DHT2 witha couple of links in the "Status" section, > http://www.gluster.org/community/documentation/index.php/Features/dht-scalability#Status > > Thanks, > Shyam > > On 01/26/2015 07:53 AM, Jeff Darcy wrote: > > Ahead of Wednesday's community meeting, I'd like to get as much 4.0 > > status together as possible. Developers, please check out the > > sub-projects on this page: > > > > http://www.gluster.org/community/documentation/index.php/Planning40 > > > > If you're involved in any of those, please update the "status" section > > of the corresponding feature page (near the bottom) as appropriate. > > Links to design notes, email threads, code review requests, etc. are > > most welcome. Yes, even things that have been sent to this list before. > > I'll be setting up a survey to choose a time for an online (Hangout/IRC) > > "summit" in the first week of February. If we can separate the items > > that are truly being worked on vs. those that just "seemed cool at the > > time" but haven't received one minute of anyone's attention since (e.g. > > sharding) then we can keep that meeting focused and productive. Thanks! > > ___ > > Gluster-devel mailing list > > Gluster-devel@gluster.org > > http://www.gluster.org/mailman/listinfo/gluster-devel > > > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [Gluster-users] v3.6.2
On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote: On 01/26/2015 09:41 PM, Justin Clift wrote: On 26 Jan 2015, at 14:50, David F. Robinson wrote: I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) I believe this error message comes when the socket file is not present. I see the following commit which changed the location of the sockets. May be Atin may know more. about this: +Atin. Ignore this mail above. I see that the commit is only present on Master: http://review.gluster.org/9423 Pranith Pranith ^C Also, when I try to NFS mount my gluster volume, I am getting Any chance there's a network or host based firewall stopping some of the ports? + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-users mailing list gluster-us...@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] v3.6.2
On 01/26/2015 09:41 PM, Justin Clift wrote: On 26 Jan 2015, at 14:50, David F. Robinson wrote: I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) I believe this error message comes when the socket file is not present. I see the following commit which changed the location of the sockets. May be Atin may know more. about this: +Atin. Pranith ^C Also, when I try to NFS mount my gluster volume, I am getting Any chance there's a network or host based firewall stopping some of the ports? + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] GlusterFS 4.0 updates needed
Updated DHT2 witha couple of links in the "Status" section, http://www.gluster.org/community/documentation/index.php/Features/dht-scalability#Status Thanks, Shyam On 01/26/2015 07:53 AM, Jeff Darcy wrote: Ahead of Wednesday's community meeting, I'd like to get as much 4.0 status together as possible. Developers, please check out the sub-projects on this page: http://www.gluster.org/community/documentation/index.php/Planning40 If you're involved in any of those, please update the "status" section of the corresponding feature page (near the bottom) as appropriate. Links to design notes, email threads, code review requests, etc. are most welcome. Yes, even things that have been sent to this list before. I'll be setting up a survey to choose a time for an online (Hangout/IRC) "summit" in the first week of February. If we can separate the items that are truly being worked on vs. those that just "seemed cool at the time" but haven't received one minute of anyone's attention since (e.g. sharding) then we can keep that meeting focused and productive. Thanks! ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Reg. multi thread epoll NetBSD failures
On 01/23/2015 03:12 PM, Emmanuel Dreyfus wrote: Shyam wrote: Patch: http://review.gluster.org/#/c/3842/ Manu, I was not able to find the NetBSD job mentioned in the last review comment provided by you, pointers to that would help. Yes, sorry, both regression tests hang and I had to reboot the machine, hence you do not have the reports in gerrit: http://build.gluster.org/job/rackspace-netbsd7-regression-triggered/731 http://build.gluster.org/job/rackspace-netbsd7-regression-triggered/741 Since I got the same horrible result, I did not try retriggering one more time, feel free to try it if you need to. What is the support status of epoll on NetBSD? I though NetBSD favored the kqueue means of event processing over epoll and that epoll was not supported on NetBSD (or *BSD). No support for epoll on NetBSD. The alternative would indeed be kqueue, but not code has been written to support it. glusterfs on NetBSD uses plain poll code right now. I ask this, as this patch specifically changes the number of epoll threads, as a result, it is possibly having a different affect on NetBSD, which should either be on poll or kqueue (to my understanding). I have not looked at the reasons right now, but it must be something outside of epoll code since NetBSD does not use it. Various tests on Linux constantly exercise the epoll option, and there are changes in the socket and poll code in this commit as well. I would assume something here causing a problem, unless others have run this through a poll mechanism. Here is what I indent to do, - Run with epoll disabled (or poll enabled) on my local box, to see if any surprises crop up - Try the same on the regression suites on gerrit (after some of the more recent runs are done) Manu, request you to go through the non-epoll changes in the submitted patch to see if something catches your eye? Shyam ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] RFC: d_off encoding at client/protocol layer
Hi, Some parts of this topic has been discussed in the recent past here [1] The current mechanism of each xlator encoding the subvol in the lower or higher bits has its pitfalls as discussed in the threads and in this review, here [2] Here is a solution design from the one of the comments posted on this by Avati here, [3], as in, "One example approach (not necessarily the best): Make every xlator knows the total number of leaf xlators (protocol/clients), and also the number of all leaf xlators from each of its subvolumes. This way, the protocol/client xlators (alone) do the encoding, by knowing its global brick# and total #of bricks. The cluster xlators blindly forward the readdir_cbk without any further transformations of the d_offs, and also route the next readdir(old_doff) request to the appropriate subvolume based on the weighted graph (of counts of protocol/clients in the subtrees) till it reaches the right protocol/client to resume the enumeration." So the current proposed scheme that is being worked on is as follows, - encode the d_off with the client/protocol ID, which is generated as its leaf position/number - no further encoding in any other xlator - on receiving further readdir requests with the d_off, consult the, graph/or immediate children, on ID encoded in the d_off, and send the request down that subvol path IOW, given a d_off and a common routine, pass the d_off with this (i.e current xlator) to get a subvol that the d_off belongs to. This routine would decode the d_off for the leaf ID as encoded in the client/protocol layer, and match its subvol relative to this and send that for further processing. (it may consult the graph or store the range of IDs that any subvol has w.r.t client/protocol and deliver the result appropriately). Given the current situation of ext4 and xfs, and continuing with the ID encoding scheme, this seems to be the best manner of preventing multiple encoding of subvol stomping on each other, and also preserving (in a sense) further loss of bits. This scheme would also give AFR/EC the ability to load balance readdir requests across its subvols better, than have a static subvol to send to for a longer duration. Thoughts/comments? Shyam [1] https://www.mail-archive.com/gluster-devel@gluster.org/msg02834.html [2] review.gluster.org/#/c/8201/4/xlators/cluster/afr/src/afr-dir-read.c [3] https://www.mail-archive.com/gluster-devel@gluster.org/msg02847.html ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [Gluster-users] v3.6.2
Tried that... Still having errors starting gluster NFS... From the /var/log/gluster/nfs.log file: [2015-01-26 19:51:25.996481] I [MSGID: 100030] [glusterfsd.c:2018:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.6.2 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket) [2015-01-26 19:51:26.005501] I [rpcsvc.c:2142:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 16 [2015-01-26 19:51:26.054144] E [nlm4.c:2481:nlm4svc_init] 0-nfs-NLM: unable to start /sbin/rpc.statd [2015-01-26 19:51:26.054183] E [nfs.c:1342:init] 0-nfs: Failed to initialize protocols [2015-01-26 19:51:26.054191] E [xlator.c:425:xlator_init] 0-nfs-server: Initialization of volume 'nfs-server' failed, review your volfile again [2015-01-26 19:51:26.054198] E [graph.c:322:glusterfs_graph_init] 0-nfs-server: initializing translator failed [2015-01-26 19:51:26.054205] E [graph.c:525:glusterfs_graph_activate] 0-graph: init failed [2015-01-26 19:51:26.05] W [glusterfsd.c:1194:cleanup_and_exit] (--> 0-: received signum (0), shutting down -- Original Message -- From: "Anatoly Pugachev" To: "David F. Robinson" Cc: "gluster-us...@gluster.org" ; "Gluster Devel" Sent: 1/26/2015 2:48:08 PM Subject: Re: [Gluster-users] v3.6.2 David, can you stop glusterfs on affected machine and remove gluster related socket extension files from /var/run ? Start glusterfs service again and try once more ? On Mon, Jan 26, 2015 at 5:57 PM, David F. Robinson wrote: Tried shutting down glusterd and glusterfsd and restarting. [2015-01-26 14:52:53.548330] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.549763] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.551245] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.552819] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.554289] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.555769] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.564429] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.565578] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/0cdef7faa934cfe52676689ff8c0110f.socket failed (Invalid argument) [2015-01-26 14:52:53.566488] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Software_bkp has disconnected from glusterd. [2015-01-26 14:52:53.567453] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/09e734d5e8d52bb796896c7a33d0a3ff.socket failed (Invalid argument) [2015-01-26 14:52:53.568248] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Software_bkp has disconnected from glusterd. [2015-01-26 14:52:53.569009] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/3f6844c74682f39fa7457082119628c5.socket failed (Invalid argument) [2015-01-26 14:52:53.569851] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Source_bkp has disconnected from glusterd. [2015-01-26 14:52:53.570818] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/34d5cc70aba63082bbb467ab450bd08b.socket failed (Invalid argument) [2015-01-26 14:52:53.571777] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Source_bkp has disconnected from glusterd. [2015-01-26 14:52:53.572681] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/0cd747876dca36cb21ecc7a36f7f897c.socket failed (Invalid argument) [2015-01-26 14:52:53.573533] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs_bkp has disconnected from glusterd. [2015-01-26 14:52:53.574433] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/88744e1365b414d41e720e480700716a.socket failed (Invalid argument) [2015-01-26 14:52:53.575399] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs_bkp has disconnected from glusterd. [2015-01-26 14:52:53.575434] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:52:53.575447] I [MSGID: 106006] [glusterd-ha
Re: [Gluster-devel] [Gluster-users] v3.6.2
[root@gfs01bkp bricks]# ps -ef | grep rpcbind rpc 2306 1 0 11:32 ?00:00:00 rpcbind root 5265 4638 0 11:55 pts/000:00:00 grep rpcbind -- Original Message -- From: "Joe Julian" To: "David F. Robinson" ; "gluster-us...@gluster.org" ; "Gluster Devel" Sent: 1/26/2015 11:55:09 AM Subject: Re: [Gluster-users] v3.6.2 Is rpcbind running? On January 26, 2015 6:57:44 AM PST, "David F. Robinson" wrote: Tried shutting down glusterd and glusterfsd and restarting. [2015-01-26 14:52:53.548330] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.549763] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.551245] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.552819] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.554289] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.555769] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.564429] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.565578] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/0cdef7faa934cfe52676689ff8c0110f.socket failed (Invalid argument) [2015-01-26 14:52:53.566488] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Software_bkp has disconnected from glusterd. [2015-01-26 14:52:53.567453] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/09e734d5e8d52bb796896c7a33d0a3ff.socket failed (Invalid argument) [2015-01-26 14:52:53.568248] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Software_bkp has disconnected from glusterd. [2015-01-26 14:52:53.569009] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/3f6844c74682f39fa7457082119628c5.socket failed (Invalid argument) [2015-01-26 14:52:53.569851] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Source_bkp has disconnected from glusterd. [2015-01-26 14:52:53.570818] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/34d5cc70aba63082bbb467ab450bd08b.socket failed (Invalid argument) [2015-01-26 14:52:53.571777] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Source_bkp has disconnected from glusterd. [2015-01-26 14:52:53.572681] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/0cd747876dca36cb21ecc7a36f7f897c.socket failed (Invalid argument) [2015-01-26 14:52:53.573533] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs_bkp has disconnected from glusterd. [2015-01-26 14:52:53.574433] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/88744e1365b414d41e720e480700716a.socket failed (Invalid argument) [2015-01-26 14:52:53.575399] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs_bkp has disconnected from glusterd. [2015-01-26 14:52:53.575434] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:52:53.575447] I [MSGID: 106006] [glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify] 0-management: nfs has disconnected from glusterd. [2015-01-26 14:52:53.579663] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick01bkp/homegfs_bkp on port 49152 [2015-01-26 14:52:53.581943] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick02bkp/Source_bkp on port 49156 [2015-01-26 14:52:53.583487] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick01bkp/Source_bkp on port 49153 [2015-01-26 14:52:53.584921] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick02bkp/Software_bkp on port 49157 [2015-01-26 14:52:53.585719] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick01bkp/Software_bkp on port 49154 [2015-01-26 14:52:53.586281] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick02bkp/homegfs_bkp on port 49155 -- Original Message -- From: "David F. Robinson" To: "gluster-us...@gluster.org" ; "Gluster Devel" Sent: 1/26/2015 9:50:09 AM Subject: v3.6.2 I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/
Re: [Gluster-devel] [Gluster-users] v3.6.2
Is rpcbind running? On January 26, 2015 6:57:44 AM PST, "David F. Robinson" wrote: >Tried shutting down glusterd and glusterfsd and restarting. > >[2015-01-26 14:52:53.548330] I >[rpc-clnt.c:969:rpc_clnt_connection_init] >0-management: setting frame-timeout to 600 >[2015-01-26 14:52:53.549763] I >[rpc-clnt.c:969:rpc_clnt_connection_init] >0-management: setting frame-timeout to 600 >[2015-01-26 14:52:53.551245] I >[rpc-clnt.c:969:rpc_clnt_connection_init] >0-management: setting frame-timeout to 600 >[2015-01-26 14:52:53.552819] I >[rpc-clnt.c:969:rpc_clnt_connection_init] >0-management: setting frame-timeout to 600 >[2015-01-26 14:52:53.554289] I >[rpc-clnt.c:969:rpc_clnt_connection_init] >0-management: setting frame-timeout to 600 >[2015-01-26 14:52:53.555769] I >[rpc-clnt.c:969:rpc_clnt_connection_init] >0-management: setting frame-timeout to 600 >[2015-01-26 14:52:53.564429] I >[rpc-clnt.c:969:rpc_clnt_connection_init] >0-management: setting frame-timeout to 600 >[2015-01-26 14:52:53.565578] W [socket.c:611:__socket_rwv] >0-management: >readv on /var/run/0cdef7faa934cfe52676689ff8c0110f.socket failed >(Invalid argument) >[2015-01-26 14:52:53.566488] I [MSGID: 106005] >[glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: >Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Software_bkp has >disconnected from glusterd. >[2015-01-26 14:52:53.567453] W [socket.c:611:__socket_rwv] >0-management: >readv on /var/run/09e734d5e8d52bb796896c7a33d0a3ff.socket failed >(Invalid argument) >[2015-01-26 14:52:53.568248] I [MSGID: 106005] >[glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: >Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Software_bkp has >disconnected from glusterd. >[2015-01-26 14:52:53.569009] W [socket.c:611:__socket_rwv] >0-management: >readv on /var/run/3f6844c74682f39fa7457082119628c5.socket failed >(Invalid argument) >[2015-01-26 14:52:53.569851] I [MSGID: 106005] >[glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: >Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Source_bkp has >disconnected from glusterd. >[2015-01-26 14:52:53.570818] W [socket.c:611:__socket_rwv] >0-management: >readv on /var/run/34d5cc70aba63082bbb467ab450bd08b.socket failed >(Invalid argument) >[2015-01-26 14:52:53.571777] I [MSGID: 106005] >[glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: >Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Source_bkp has >disconnected from glusterd. >[2015-01-26 14:52:53.572681] W [socket.c:611:__socket_rwv] >0-management: >readv on /var/run/0cd747876dca36cb21ecc7a36f7f897c.socket failed >(Invalid argument) >[2015-01-26 14:52:53.573533] I [MSGID: 106005] >[glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: >Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs_bkp has >disconnected from glusterd. >[2015-01-26 14:52:53.574433] W [socket.c:611:__socket_rwv] >0-management: >readv on /var/run/88744e1365b414d41e720e480700716a.socket failed >(Invalid argument) >[2015-01-26 14:52:53.575399] I [MSGID: 106005] >[glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: >Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs_bkp has >disconnected from glusterd. >[2015-01-26 14:52:53.575434] W [socket.c:611:__socket_rwv] >0-management: >readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed >(Invalid argument) >[2015-01-26 14:52:53.575447] I [MSGID: 106006] >[glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify] 0-management: >nfs has disconnected from glusterd. >[2015-01-26 14:52:53.579663] I [glusterd-pmap.c:227:pmap_registry_bind] > >0-pmap: adding brick /data/brick01bkp/homegfs_bkp on port 49152 >[2015-01-26 14:52:53.581943] I [glusterd-pmap.c:227:pmap_registry_bind] > >0-pmap: adding brick /data/brick02bkp/Source_bkp on port 49156 >[2015-01-26 14:52:53.583487] I [glusterd-pmap.c:227:pmap_registry_bind] > >0-pmap: adding brick /data/brick01bkp/Source_bkp on port 49153 >[2015-01-26 14:52:53.584921] I [glusterd-pmap.c:227:pmap_registry_bind] > >0-pmap: adding brick /data/brick02bkp/Software_bkp on port 49157 >[2015-01-26 14:52:53.585719] I [glusterd-pmap.c:227:pmap_registry_bind] > >0-pmap: adding brick /data/brick01bkp/Software_bkp on port 49154 >[2015-01-26 14:52:53.586281] I [glusterd-pmap.c:227:pmap_registry_bind] > >0-pmap: adding brick /data/brick02bkp/homegfs_bkp on port 49155 > > > >-- Original Message -- >From: "David F. Robinson" >To: "gluster-us...@gluster.org" ; "Gluster >Devel" >Sent: 1/26/2015 9:50:09 AM >Subject: v3.6.2 > >>I have a server with v3.6.2 from which I cannot mount using NFS. The >>FUSE mount works, however, I cannot get the NFS mount to work. From >>/var/log/message: >> >>Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server >>'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying >>Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server >>'gfsib01bkp.corvidtec.com' failed: Connection refused, retryin
Re: [Gluster-devel] v3.6.2
No firewall used on that machine. [root@gfs01bkp ~]# /etc/init.d/iptables status iptables: Firewall is not running. [ [root@gfs01bkp ~]# cat /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted -- Original Message -- From: "Justin Clift" To: "David F. Robinson" Cc: "Gluster Users" ; "Gluster Devel" Sent: 1/26/2015 11:11:15 AM Subject: Re: [Gluster-devel] v3.6.2 On 26 Jan 2015, at 14:50, David F. Robinson wrote: I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) ^C Also, when I try to NFS mount my gluster volume, I am getting Any chance there's a network or host based firewall stopping some of the ports? + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] v3.6.2
On 26 Jan 2015, at 14:50, David F. Robinson wrote: > I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE > mount works, however, I cannot get the NFS mount to work. From > /var/log/message: > > Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server > 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying > > > I also am continually getting the following errors in /var/log/glusterfs: > > [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log > [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] 0-management: > readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid > argument) > ^C > > Also, when I try to NFS mount my gluster volume, I am getting Any chance there's a network or host based firewall stopping some of the ports? + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] v3.6.2
Tried shutting down glusterd and glusterfsd and restarting. [2015-01-26 14:52:53.548330] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.549763] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.551245] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.552819] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.554289] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.555769] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.564429] I [rpc-clnt.c:969:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2015-01-26 14:52:53.565578] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/0cdef7faa934cfe52676689ff8c0110f.socket failed (Invalid argument) [2015-01-26 14:52:53.566488] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Software_bkp has disconnected from glusterd. [2015-01-26 14:52:53.567453] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/09e734d5e8d52bb796896c7a33d0a3ff.socket failed (Invalid argument) [2015-01-26 14:52:53.568248] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Software_bkp has disconnected from glusterd. [2015-01-26 14:52:53.569009] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/3f6844c74682f39fa7457082119628c5.socket failed (Invalid argument) [2015-01-26 14:52:53.569851] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/Source_bkp has disconnected from glusterd. [2015-01-26 14:52:53.570818] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/34d5cc70aba63082bbb467ab450bd08b.socket failed (Invalid argument) [2015-01-26 14:52:53.571777] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/Source_bkp has disconnected from glusterd. [2015-01-26 14:52:53.572681] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/0cd747876dca36cb21ecc7a36f7f897c.socket failed (Invalid argument) [2015-01-26 14:52:53.573533] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs_bkp has disconnected from glusterd. [2015-01-26 14:52:53.574433] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/88744e1365b414d41e720e480700716a.socket failed (Invalid argument) [2015-01-26 14:52:53.575399] I [MSGID: 106005] [glusterd-handler.c:4142:__glusterd_brick_rpc_notify] 0-management: Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs_bkp has disconnected from glusterd. [2015-01-26 14:52:53.575434] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:52:53.575447] I [MSGID: 106006] [glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify] 0-management: nfs has disconnected from glusterd. [2015-01-26 14:52:53.579663] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick01bkp/homegfs_bkp on port 49152 [2015-01-26 14:52:53.581943] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick02bkp/Source_bkp on port 49156 [2015-01-26 14:52:53.583487] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick01bkp/Source_bkp on port 49153 [2015-01-26 14:52:53.584921] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick02bkp/Software_bkp on port 49157 [2015-01-26 14:52:53.585719] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick01bkp/Software_bkp on port 49154 [2015-01-26 14:52:53.586281] I [glusterd-pmap.c:227:pmap_registry_bind] 0-pmap: adding brick /data/brick02bkp/homegfs_bkp on port 49155 -- Original Message -- From: "David F. Robinson" To: "gluster-us...@gluster.org" ; "Gluster Devel" Sent: 1/26/2015 9:50:09 AM Subject: v3.6.2 I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec
[Gluster-devel] v3.6.2
I have a server with v3.6.2 from which I cannot mount using NFS. The FUSE mount works, however, I cannot get the NFS mount to work. From /var/log/message: Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying I also am continually getting the following errors in /var/log/glusterfs: [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument) ^C Also, when I try to NFS mount my gluster volume, I am getting === David F. Robinson, Ph.D. President - Corvid Technologies 704.799.6944 x101 [office] 704.252.1310 [cell] 704.799.7974 [fax] david.robin...@corvidtec.com http://www.corvidtechnologies.com ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Netbsd regrsssion needs review
rJustin Clift wrote: > > Justin, we will need to tell jenkins we want to run Linux and NetBSD > > regression in parallel. Do you have an idea on how it can be configured? > > Not sure yet. My head isn't in Jenkins-mode atm, and probably won't be > until I get back from FOSDEM (leaving today, returning on 4th Feb). ;) Well, give it a try: we already run grouped tests for smoke builds. The idea is to do the same for regression. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Netbsd regrsssion needs review
On 26 Jan 2015, at 13:57, Emmanuel Dreyfus wrote: > Justin Clift wrote: > >> Anyone? > > Welcome back :-) Thanks. :) > Here are the latest suprious regression tests fixes for NetBSD: > http://review.gluster.org/9483 > http://review.gluster.org/9461 > http://review.gluster.org/9074 Cool. Lets try and get these merged today/tomorrow... I'm not sure who's going to be at FOSDEM and its associated events, which may or may not affect who's around tho. ;) > After these ones, we still have a rare failure in ec-12-4.t which > happens around 70 runs. I think we will be able to enable NetBSD > regression voting before fixing it. Yeah, that one sounds rare enough to allow it through for now. :) > Justin, we will need to tell jenkins we want to run Linux and NetBSD > regression in parallel. Do you have an idea on how it can be configured? Not sure yet. My head isn't in Jenkins-mode atm, and probably won't be until I get back from FOSDEM (leaving today, returning on 4th Feb). ;) + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Netbsd regrsssion needs review
Justin Clift wrote: > Anyone? Welcome back :-) Here are the latest suprious regression tests fixes for NetBSD: http://review.gluster.org/9483 http://review.gluster.org/9461 http://review.gluster.org/9074 After these ones, we still have a rare failure in ec-12-4.t which happens around 70 runs. I think we will be able to enable NetBSD regression voting before fixing it. Justin, we will need to tell jenkins we want to run Linux and NetBSD regression in parallel. Do you have an idea on how it can be configured? -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Netbsd regrsssion needs review
Anyone? On 23 Jan 2015, at 13:37, Emmanuel Dreyfus wrote: > Hello > > Can someone please have a look at these. Most netbsd regression failures > are caused by spurious failures fixed there. > http://review.gluster.org/9461 (only fix the test script) > http://review.gluster.org/9074(this one probably needs a discussion) > > I can add a third one fixed this morning: > http://review.gluster.org/9483 (only fixes the test script) > > And while I am there, two backports: > http://review.gluster.org/9448 > http://review.gluster.org/9484 > > On Tue, Jan 20, 2015 at 05:28:24AM +0100, Emmanuel Dreyfus wrote: >> NetBSD regression tests suffers a few spurious failures. I have fixes >> for the two worst offenders: >> http://review.gluster.org/9461 >> http://review.gluster.org/9074 >> >> Can someone please review? > > -- > Emmanuel Dreyfus > m...@netbsd.org > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] GlusterFS 4.0 updates needed
Ahead of Wednesday's community meeting, I'd like to get as much 4.0 status together as possible. Developers, please check out the sub-projects on this page: http://www.gluster.org/community/documentation/index.php/Planning40 If you're involved in any of those, please update the "status" section of the corresponding feature page (near the bottom) as appropriate. Links to design notes, email threads, code review requests, etc. are most welcome. Yes, even things that have been sent to this list before. I'll be setting up a survey to choose a time for an online (Hangout/IRC) "summit" in the first week of February. If we can separate the items that are truly being worked on vs. those that just "seemed cool at the time" but haven't received one minute of anyone's attention since (e.g. sharding) then we can keep that meeting focused and productive. Thanks! ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [Gluster-users] lockd: server not responding, timed out
On Mon, Jan 26, 2015 at 12:26:53AM +, Peter Auyeung wrote: > Hi Niels, > > The question if we keep getting the lockd error even after restart and > rebooted the NFS client.. This particular error would only occur when the NFS-server could not register the nlockmgr RPC-program to rpcbind/portmapper. The most likely scenario where this fails, is where there is an NFS-client (or service) on the storage server that conflicts with the Gluster/NFS service. If there are conflicting RPC services in rpcbind/portmapper, you may be able check and remove those with the 'rpcinfo' command. Ports that are listed in te output, but are not listed in netstat/ss are in used by kernel services (like the lockd kernel module). In order to restore the NLM function of Gluster/NFS, you can take these steps: 1. ensure that there are no other NFS-services (server or client) running on the Gluster storage server. Gluster/NFS should be the only service which does some NFS on the server. 2. stop the rpcbind service 3. clear the rpcbind-cache (rm /var/lib/rpcbind/portmap.xdr) 4. start the rpcbind service 5. restart the Gluster/NFS service In case your NFS-client got connected to the incorrect NLM service on your storage server, you would need to unmount and mount the export again. Niels > > Peter > > From: Niels de Vos [nde...@redhat.com] > Sent: Saturday, January 24, 2015 3:26 AM > To: Peter Auyeung > Cc: gluster-us...@gluster.org; gluster-devel@gluster.org > Subject: Re: [Gluster-devel] [Gluster-users] lockd: server not responding, > timed out > > On Fri, Jan 23, 2015 at 11:50:26PM +, Peter Auyeung wrote: > > We have a 6 nodes gluster running ubuntu on xfs sharing gluster > > volumes over NFS been running fine for 3 months. > > We restarted glusterfs-server on one of the node and all NFS clients > > start getting the " lockd: server not responding, timed out" on > > /var/log/messages > > > > We are still able to read write but seems like process that require a > > persistent file lock failed like database exports. > > > > We have an interim fix to remount the NFS with nolock option but need > > to know why that is necessary all in a sudden after a service > > glusterfs-server restart on one of the gluster node > > The cause that you need to mount wiht 'nolock' is that one server can > only have one NLM-service active. The Linux NFS-client uses the 'lockd' > kernel module, and the Gluster/NFS server provides its own lock manager. > To be able to use a lock manager, it needs to be registered at > rpcbind/portmapper. Only one lock manager can be registered at a time, > the 2nd one that tries to register will fail. In case the NFS-client has > registered the lockd kernel module as lock manager, any locking requests > to the Gluster/NFS service will fail and you will see those messages in > /var/log/messages. > > This is one of the main reasons why it is not advised to access volumes > over NFS on a Gluster storage server. You should rather use the > GlusterFS protocol for mounting volumes locally. (Or even better, > seperate your storage servers from the application servers.) > > HTH, > Niels pgphd80mfXt8M.pgp Description: PGP signature ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Ability to turn off 'glusterfs' protocol
On Sun, Jan 25, 2015 at 10:08:20PM +0530, Ramana Raja wrote: > On Wed, Jan 21, 2015 at 8:42 AM, Vijay Bellur wrote: > > > Given this, we could implement this feature by serving volfiles to only > > trusted clients in glusterd and fail requests from everywhere else if an > > option to disable glusterfs protocol has been set. This way all services > > accessing volumes locally from the trusted storage pool will continue to > > function without any problems. > > > > > I've a couple of questions, > > 1) How would the above suggestion impact gfapi access? gfapi uses the GlusterFS protocol. When access through the protocol is denied, only clients from within the trusted storage pool can use it. gfapi is just a client, similar to the FUSE mount. > 2) Would the list of trusted clients be configured via "gluster volume set"? There are the 'auth.allow' and 'auth.reject' volume options. I think they would be used as an alternative to a "turn off glusterfs protocol" feature. HTH, Niels pgp2gt_mhZcKx.pgp Description: PGP signature ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
