Send Gluster-users mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Gluster-users digest..."
Today's Topics:
1. Re: Reading directly from brick (Reinis Rozitis)
2. Re: NFS secondary groups not working. (Di Pe)
3. Inconsistent md5sum of replicated file (Anthony Delviscio)
4. Re: Inconsistent md5sum of replicated file (Pranith Kumar K)
5. Problems with SLES 11 (Phil Bayfield)
----------------------------------------------------------------------
Message: 1
Date: Tue, 6 Sep 2011 23:24:24 +0300
From: "Reinis Rozitis"<[email protected]>
Subject: Re: [Gluster-users] Reading directly from brick
To:<[email protected]>
Message-ID:<F7DAC991835C44889BCDB281F977B692@NeiRoze>
Content-Type: text/plain; format=flowed; charset="utf-8";
reply-type=original
Simple answer - no, it's not ever safe to do writes to an active Gluster
backend.
Question was about reads though and then the answer is it is perfectly fine
(and faster) to do reads directly from the filesystem (in replicated setups)
if you keep in mind that by doing so you lose the Glusters autoheal
eature - eg if one of the gluster nodes goes down and there is a file
written meanwhile when the server comes up if you access the file directly
it won't show up while it would when accessing it via the gluster mount
point (you can work arround it by manually triggering the self heal).
I've heard that reads from glusterfs are around 20 times slower than from
ext3:
"20 times" might be fetched out of thin air but of course there is a
significant overhead of serving a file from a gluster which basically
involves network operations and additional meta data checks versus fetching
the file directly from iron.
rr
------------------------------
Message: 2
Date: Tue, 6 Sep 2011 14:46:28 -0700
From: Di Pe<[email protected]>
Subject: Re: [Gluster-users] NFS secondary groups not working.
To: [email protected]
Message-ID:
<cab9t+o+fab+yasvxmsusvmmw0scp3blsqc0y_grusrmv11q...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Anand, has this issue been confirmed by gluster and is it in the pipe
to get fixed or do you need .additional information? We are no gluster
experts but are happy to help if we know who to provide additional
debugging info.
On Mon, Aug 29, 2011 at 9:44 AM, Mike Hanby<[email protected]> wrote:
I just noticed the problem happening on one client in our environment (clients
and servers running 3.2.2), other clients work fine.
The clients and servers are all CentOS 5.6 x86_64
I get the same permission denied using Gluster FUSE and Gluster NFS mounts on
this client.
I'm not mounting it with ACL.
The volume is a simple distributed volume with two servers.
-----Original Message-----
From: [email protected] [mailto:gluster-users-
[email protected]] On Behalf Of Hubert-Jan Schaminee
Sent: Saturday, August 27, 2011 10:10 AM
To: Anand Avati
Cc: [email protected]
Subject: Re: [Gluster-users] NFS secondary groups not working.
Op zaterdag 13-08-2011 om 20:22 uur [tijdzone +0530], schreef Anand
Avati:
On Sat, Aug 13, 2011 at 5:29 PM, Dipeit<[email protected]> wrote:
? ? ? ? We noticed this bug too using the gluster client. I'm
? ? ? ? surprised that not more people noticed this lack of posix
? ? ? ? compliance. This makes gluster really unusable in multiuser
? ? ? ? environments. Is that because gluster is mostly used in large
? ? ? ? web farms like pandora?
GlusterFS is POSIX compliant w.r.t user groups. We have not seen this
issue in our testing. Can you give more info about your setup? Have
you mounted with -o acl or without? Anything unusual in the logs?
Avati
I'm having the same problem here.
I use the latest version (3.2.3 build on Aug 23 2011 19:54:51 of the
download site) on a Centos 5.6 as a gluster servers, Debian squeeze
(same version) as client.
I'm refused access to files and directories despite having correct
group permissions.
So I installed a clean Centos client (also latest version) for a test
and everything is working perfectly .... ?
The used Debian (squeeze) and Centos are 64 bits (repository from
gluster.com).
Using Debian testing (64 and 32 bits) and gluster from the Debian
repository also denies me access in 64 and 32 bits version.
I assume the mixed environment explains why this bug is rare.
The used gluster installation is a basic replicated setup one with two
servers like described in the de Gluster docs.
Hubert-Jan Schamin?e
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
------------------------------
Message: 3
Date: Tue, 6 Sep 2011 18:52:52 -0400
From: Anthony Delviscio<[email protected]>
Subject: [Gluster-users] Inconsistent md5sum of replicated file
To: [email protected]
Message-ID:
<cake0inqy3tjf3tb11kc+f_f-p7kn2cj+eg+2faruxoe4tnz...@mail.gmail.com>
Content-Type: text/plain; charset="windows-1252"
I was wondering if anyone would be able to shed some light on how a file
could end up with inconsistent md5sums on Gluster backend storage.
Our configuration is running on Gluster v3.1.5 in a distribute-replicate
setup consisting of 8 bricks.
Our OS is Red Hat 5.6 x86_64. Backend storage is an ext3 RAID 5.
The 8 bricks are in RR DNS and are mounted for reading/writing via NFS
automounts.
When comparing md5sums of the file from two different NFS clients, they were
different.
The extended attributes of the files on backend storage are identical. The
file size and permissions are identical. The stat data (excluding inode on
backend storage file system) is identical.
However, running md5sum on the two files, results in two different md5sums.
Copying both files to another location/server and running the md5sum also
results in no change ? they?re still different.
Gluster logs do not show anything related to the filename in question.
Triggering
a self-healing operation didn?t seem to do anything and it may have to do
with the fact that the extended attributes are identical.
If more information is required, let me know and I will try to accommodate.
Thank you
-------------- next part --------------
An HTML attachment was scrubbed...
URL:<http://gluster.org/pipermail/gluster-users/attachments/20110906/4628faa2/attachment-0001.htm>
------------------------------
Message: 4
Date: Wed, 7 Sep 2011 14:13:56 +0530
From: Pranith Kumar K<[email protected]>
Subject: Re: [Gluster-users] Inconsistent md5sum of replicated file
To: Anthony Delviscio<[email protected]>
Cc: [email protected]
Message-ID:<[email protected]>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"
hi Anthony,
Could you send the output of the getfattr -d -m . -e hex
<filepath> on both the bricks and also the stat output on the both the
backends. Give the outputs for its parent directory also.
Pranith.
On 09/07/2011 04:22 AM, Anthony Delviscio wrote:
I was wondering if anyone would be able to shed some light on how a
file could end up with inconsistent md5sums on Gluster backend storage.
Our configuration is running on Gluster v3.1.5 in a
distribute-replicate setup consisting of 8 bricks.
Our OS is Red Hat 5.6 x86_64.Backend storage is an ext3 RAID 5.
The 8 bricks are in RR DNS and are mounted for reading/writing via NFS
automounts.
When comparing md5sums of the file from two different NFS clients,
they were different.
The extended attributes of the files on backend storage are
identical.The file size and permissions are identical.The stat data
(excluding inode on backend storage file system) is identical.
However, running md5sum on the two files, results in two different
md5sums.
Copying both files to another location/server and running the md5sum
also results in no change ? they?re still different.
Gluster logs do not show anything related to the filename in
question.Triggering a self-healing operation didn?t seem to do
anything and it may have to do with the fact that the extended
attributes are identical.
If more information is required, let me know and I will try to
accommodate.
Thank you
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:<http://gluster.org/pipermail/gluster-users/attachments/20110907/86d14cab/attachment-0001.htm>
------------------------------
Message: 5
Date: Wed, 7 Sep 2011 10:15:43 +0100
From: Phil Bayfield<[email protected]>
Subject: [Gluster-users] Problems with SLES 11
To: [email protected]
Message-ID:
<cafxh-fw0dbe9yomjzatvdfawaf5zpq-tfbftpb+k7gbu-r+...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi there,
I compiled and installed the latest version of Gluster on a couple of SLES
11 SP1 boxes, everything up to this point seemed ok.
I start the daemon on both boxes, and both are listening on 24007.
I issue a "gluster peer probe" command on one of the boxes and the daemon
instantly dies, I restart it and it shows:
# gluster peer status
Number of Peers: 1
Hostname: mckalcpap02
Uuid: 00000000-0000-0000-0000-000000000000
State: Establishing Connection (Connected)
I attempted to run the probe on the other box, the daemon crashes, now as I
start the daemon on each box the daemon just crashes on the other box.
The log output immediately prior to the crash is as follows:
[2011-06-07 08:05:10.700710] I
[glusterd-handler.c:623:glusterd_handle_cli_probe] 0-glusterd: Received CLI
probe req mckalcpap02 24007
[2011-06-07 08:05:10.701058] I [glusterd-handler.c:391:glusterd_friend_find]
0-glusterd: Unable to find hostname: mckalcpap02
[2011-06-07 08:05:10.701086] I
[glusterd-handler.c:3422:glusterd_probe_begin] 0-glusterd: Unable to find
peerinfo for host: mckalcpap02 (24007)
[2011-06-07 08:05:10.702832] I [glusterd-handler.c:3404:glusterd_friend_add]
0-glusterd: connect returned 0
[2011-06-07 08:05:10.703110] I
[glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using Program
glusterd clnt mgmt, Num (1238433), Version (1)
If I use the IP address the same thing happens:
[2011-06-07 08:07:12.873075] I
[glusterd-handler.c:623:glusterd_handle_cli_probe] 0-glusterd: Received CLI
probe req 10.9.54.2 24007
[2011-06-07 08:07:12.873410] I [glusterd-handler.c:391:glusterd_friend_find]
0-glusterd: Unable to find hostname: 10.9.54.2
[2011-06-07 08:07:12.873438] I
[glusterd-handler.c:3422:glusterd_probe_begin] 0-glusterd: Unable to find
peerinfo for host: 10.9.54.2 (24007)
[2011-06-07 08:07:12.875046] I [glusterd-handler.c:3404:glusterd_friend_add]
0-glusterd: connect returned 0
[2011-06-07 08:07:12.875280] I
[glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using Program
glusterd clnt mgmt, Num (1238433), Version (1)
There is no firewall issue:
# telnet mckalcpap02 24007
Trying 10.9.54.2...
Connected to mckalcpap02.
Escape character is '^]'.
Following restart (which crashes the other node) the log output is as
follows:
[2011-06-07 08:10:09.616486] I [glusterd.c:564:init] 0-management: Using
/etc/glusterd as working directory
[2011-06-07 08:10:09.617619] C [rdma.c:3933:rdma_init] 0-rpc-transport/rdma:
Failed to get IB devices
[2011-06-07 08:10:09.617676] E [rdma.c:4812:init] 0-rdma.management: Failed
to initialize IB Device
[2011-06-07 08:10:09.617700] E [rpc-transport.c:741:rpc_transport_load]
0-rpc-transport: 'rdma' initialization failed
[2011-06-07 08:10:09.617724] W [rpcsvc.c:1288:rpcsvc_transport_create]
0-rpc-service: cannot create listener, initing the transport failed
[2011-06-07 08:10:09.617830] I [glusterd.c:88:glusterd_uuid_init]
0-glusterd: retrieved UUID: 1e344f5d-6904-4d14-9be2-8f0f44b97dd7
[2011-06-07 08:10:11.258098] I [glusterd-handler.c:3404:glusterd_friend_add]
0-glusterd: connect returned 0
Given volfile:
+------------------------------------------------------------------------------+
1: volume management
2: type mgmt/glusterd
3: option working-directory /etc/glusterd
4: option transport-type socket,rdma
5: option transport.socket.keepalive-time 10
6: option transport.socket.keepalive-interval 2
7: end-volume
8:
+------------------------------------------------------------------------------+
[2011-06-07 08:10:11.258431] I
[glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using Program
glusterd clnt mgmt, Num (1238433), Version (1)
[2011-06-07 08:10:11.280533] W [socket.c:1494:__socket_proto_state_machine]
0-socket.management: reading from socket failed. Error (Transport endpoint
is not connected), peer (10.9.54.2:1023)
[2011-06-07 08:10:11.280595] W [socket.c:1494:__socket_proto_state_machine]
0-management: reading from socket failed. Error (Transport endpoint is not
connected), peer (10.9.54.2:24007)
[2011-06-07 08:10:17.256235] E [socket.c:1685:socket_connect_finish]
0-management: connection to 10.9.54.2:24007 failed (Connection refused)
There are no logs on the node which crashes.
I've tried various possibly solutions from searching the net but got getting
anywhere, can anyone advise how to proceed?
Thanks,
Phil.
