Been there ...
here is my 10cent advise
a) Prepare for tomorrow
b) Rest
c) Think
d) Plan
e) act
I am sure it will work for you when calmed
Tech hints.
ifconfig iface mtu 9000 or whatever your nic can afford
Having a 100Mbit is not a good idea.
I 've recently located a dual port 1Gbit nic on ebay for $15 USD
Get them.
And last but not least
In case you happen to have a switch between the nodes make sure that you
enable jumbo frames on it.
Otherwise you are in DEEP Trouble.
stat' ing a 120G file in my case takes miliseconds not even seconds.
GL
Harry.
On 10/07/2013 02:01 μμ, Allan Latham wrote:
Hi all
Thanks to all those volunteers who are working to get gluster into a
state where it can be used for live work.
I understand that you are giving your free time and I very much
appreciate it on this project and the many others we use for live
production work.
There seems to be a problem with the way gluster is going.
For me it would be an ideal solution if it actually worked.
I have a simple scenario and it just simply doesn't work. Reading over
the network when the file is available locally is plainly wrong. Our
application cannot take the performance hit nor the extra network traffic.
I would suggest:
1. get a simple minimalist configuration working - 2 hosts and
replication only.
2. make it bomb-proof.
2a. it must cope with network failures, random reboots etc.
2b. if it stops it has to auto-recover quickly.
2c. if it can't it needs thorough documentation and adequate logs so a
reasonable sysop can rescue it.
2d. it needs a fast validation scanner which verifies that data is where
it should be and is identical everywhere (md5sum).
3. make it efficient (read local whenever possible - use rsync
techniques - remove scalability obstacles so it doesn't get
exponentially slower as more files are replicated)
4. when that works expand to multiple hosts and clever distribution
techniques.
(repeat items 2 and 3 in the more complex environment)
If it doesn't work rock solid in a simple scenario it will never work in
a large scale cluster.
Until point 3 is reached I cannot use it - which is a great
disappointment for me as well as the good guys doing the development.
Good luck and thanks again
Allan
On 04/07/13 13:10, Allan Latham wrote:
Hi all
Does anyone use read-subvolume?
Has anyone tested read-subvolume?
Does read-subvolume work in such a way that if the file is present on
the local node the local copy is used rather than a remote one?
Alternatively is there any way to configure (or patch) gluster to always
prefer the local file?
I have read everything available and have found no answer.
Unison works very well in our environment but is not real time and needs
to be run every few minutes and/or be kicked off with inotify.
If I could get gluster to always read the local copy it would be a much
better drop in replacement.
This is a small scale deployment not a massive cluster but I can imagine
there are many potential users of gluster in this mode. It should beat
unison and similar solutions in every way - but it doesn't because it is
reading from the network even when it has a local up-to-date copy. This
can't be intended behaviour.
So what have I configured wrong?
Thanks in advance
Allan
On 02/07/13 13:38, Allan Latham wrote:
Hi everyone
I have installed 3.3.1-1 from the Debian repository you provide.
I am using a simple 2 node cluster and running in replication mode. The
connection between the nodes is limited to 100MB/sec (that's bits not
bytes!). Usage will be mainly for read access and since there is always
a local copy available [ exactly 2 replicas on exactly 2 machines ] I
expect very fast read performance. Writes are low volume and very
infrequent - performance is not an issue.
Almost everything works as I would expect.
Write speed is limited to 10Mb (bytes) per second which is what I would
expect and is adequate for the application.
But read speed is either super fast or 10Mb/sec. i.e. read operations
take place on the local copy or the remote seemingly at random.
This not the 'small files problem'. I am aware that Gluster must use
network access for stat() etc. This is all about where the data comes
from on a read(). If I do an m5dum on a 200Mb file it takes either half
a second or 18 seconds.
There is an option read-subvolume.
I have tried to understand how this works from the documentation
available and from the few examples on the web.
I have added the option using:
gluster volume set X read-subvolume Y
It has no effect even after stopping and starting the volume,
remounting, restarting gluster servers etc.
What's more I fail to see how this option could ever work at all. The
configuration changes caused by the above command are rolled out to both
nodes - but what is right for one node is exactly the wrong
configuration for the other node.
Configs attached are in /var/lib/glusterd/vols/shared except
glusterd.vol which is in /etc/glusterfs.
Here is the output of the mount command filtered to just the glusterfs
mount:
10.255.255.1:/shared on /gluster/rw/shared type fuse.glusterfs
(rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
10.255.255.1 is local to this host.
I would be very thankful if someone can enlighten me. I am obviously
configuring this wrong. I may have missed something important.
Best regards to all
Allan
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
