I'm wondering if there's some way for glusterfs to detect the flaws of
the underlying operating system. I believe there's no bug-free file
systems in the universe, so I believe it is the job of the glusterfs
developer to specify which underlying filesystem is tested and
supported. It's not good to simply say that glusterfs works on all
real-world approximations to an imaginary bug-free posix filesystem.
- Wei
Stephan von Krawczynski wrote:
On Fri, 28 Aug 2009 14:28:51 +0200
David Saez Padros <[email protected]> wrote:
Hi
well, that never hapen before when using nfs with the same
computers, same disk, etc ... for almost 2 years, so it's more
than possible that is glusterfs the one which is triggering this
suposed ext3 bug, but appart from this:
I can assure you that you will never have an agreement on this point on this
list, this happens to be the only bugfree software in universe according to
authors ;-)
a) documentation says "All operations that do not modify the file
or directory are sent to all the subvolumes and the first successful
reply is returned to the application", why is blocking then ?
it's suposed that the reply from the non blocked server will
come first and nothing will block, but clients are blocking on
a simple ls operation
My impression is that you have to imagine the setup as serialized queue on the
server. If there was one operation with a hang, all future ones will be
hanging, too.
b) server1 (the non blocked one) also has the volumes mounted like
any other client, but having option read-subvolume set to the local
volume, but it also hangs when it was suposed to read from the local
volume, not from the hanged one
This is exactly my experience. You cannot make it work either way. There seems
to be some locking across all used servers.
c) does not glsuterfs ping the servers periodically to see if they
are available or not ? if so, why does not it detect that situation ?
Well, this ping-pong procedure seems to be only detecting offline servers
(i.e. network down), but is obviously not able to give hints about being
operational or not.
My idea of a solution would be to implement something like a bail-out timeout
configurable on the client vol file for every brick. This would allow to
intermix slow and fast servers and it would cope with a situation where some
clients are far away with a slow connnections and others are nearby with very
fast connection to the same servers.
The biggest problem about it probably is not to bail out servers, but to
re-integrate them. Currently there seems to be no userspace tool to tell a
client to re-integrate a formerly dead server. Obviously this should not
happen auto-magically to prevent flapping.
[...]
Glusterfs log only shows lines like this ones:
[2009-08-28 09:19:28] E [client-protocol.c:292:call_bail] data2: bailing
out frame LOOKUP(32) frame sent = 2009-08-28 08:49:18. frame-timeout = 1800
[2009-08-28 09:23:38] E [client-protocol.c:292:call_bail] data2: bailing
out frame LOOKUP(32) frame sent = 2009-08-28 08:53:28. frame-timeout = 1800
Once server2 has been rebooted all gluster fs become available
again on all clients and the hanged df and ls processes terminate,
but difficult to understand why a replicated share that must survive
to failure on one server does not.
You are suffering from the problem we talked about few days ago on the list.
If your local fs produces a deadlock somehow on one server glusterfs is
currently unable to cope with the situation and just _waits_ for things to
come. This deadlocks your clients, too, without any need.
Your experience backs my critics on the handling of these situations.
--
Best regards ...
----------------------------------------------------------------
David Saez Padros http://www.ols.es
On-Line Services 2000 S.L. telf +34 902 50 29 75
----------------------------------------------------------------
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
