Mark Mielke wrote:
Possibly relevant here -
At work, we have used a tool which does something similar to
booster to accelerate an extremely slow remote file system. It
works the same way with LD_PRELOAD, however, it also requires GLIBC
to be compiled with --disable-hidden-plt. Reviewing the Internet
for similar solutions, will find PlasticFS which also has the same
requirement.
Recent versions of GLIBC call open() internally without following
the regular the regular PLT name resolution model. This increases
performance as the PLT indirect lookup model has an expense
associated. For example, GLIBC fopen() calls open() directly rather
than going through the PLT. So, overriding open() does not
intercept calls to fopen()?
Is this something the booster developers are aware of? Have they
found a way around this, or is it possible that booster is only
boosting *some* types of access, and other types of access are
still "falling through" to FUSE?
I've asked the developer who wrote out library what he thought of
glusterfs/booster not requiring GLIBC with --disable-hidden-plt,
and he thinks glusterfs/booster cannot be working (or cannot be
intercepting all calls and some calls are leaking through to FUSE).
Comments?
If some calls were leaking through, this might have the "double
traffic" effect, since FUSE would have its own cache separate from
booster?
I dont know what a PLT is but I'll attempt to provide some clarity here.
It is true, that booster does not support or boost all system calls.
We do not require that glibc be built with --disable-hidden-plt
for those calls which we do support.
For a start, we've aimed at getting apache and unfs3 to work with
booster. The functional support for both in booster is complete in
2.0.6 release.
For a list of system calls supported by booster, please see:
http://www.gluster.org/docs/index.php/BoosterConfiguration
There can be applications which need un-boosted syscalls also to be
usable over GlusterFS. For such a scenario we have two ways booster
can be used. Both approaches are described at the page linked above
but in short, you're right in thinking that when the un-supported
syscalls are also needed to go over FUSE, we are, as you said, leaking
or redirecting calls over the FUSE mount point.
That page is a bit long so feel free to ask any questions here.
Thanks
-Shehjar
Cheers, mark
On 08/14/2009 01:22 PM, Anand Avati wrote:
I've been running 2.0.3 with two backend bricks and a frontend
client of mod_gluster/apache 2.2.11+worker for a few weeks now
without much issue. Last night i upgraded to 2.0.6 only to find
out that mod_gluster has been removed and is recommending to
use the booster library - which is fine but i didnt have time
to test it last night so i just mounted the whole filesystem
with a fuse mount and figured id test the booster config later
and then swap. I did try running the 2.0.3 mod_gluster module
with the 2.0.6 bricks but apache kept segfaulting (every 10
seconds) and then would spawn another process which would
reconnect and keep going. I figured it was dropping a client
request every few seconds which is why i went with the fuse
mount until i could test the booster library.
That would not work, swapping binaries across versions.
Well, before with mod_gluster, we would be pushing around
200mbit of web traffic and it would evenly distribute that
200mbit between our two bricks - so server1 would be pushing
100mbit and server2 would be pushing another 100mbit.
Basically both inbound from the backend bricks and outbound
from apache was basically identical. Except of course if one
of the backend glusterd processes died for whatever reason the
other remaining brick would take the whole load and its traffic
would double as you would expect. Perfect, all was happy.
Now using gluster 2.0.6 and fuse both server bricks are pushing
the full 200mbit of traffic - so i basically have 400mbit of
incoming traffic from the gluster bricks but the same 200mbit
of web traffic. I can deal, but i only have a shared gigabit
link between my client server and backend bricks and im already
eating up basically 50% of that pipe. It is also putting a
much larger load on both bricks since i have basically doubled
the disk IO time and traffic. Is this a feature? Bug?
If I understand correct, 2.0.3 mod_glusterfs = 1x, 2.0.6 fuse =
2x? Can you describe the files being served? (average file size
and number of files)
Avati _______________________________________________
Gluster-users mailing list Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
------------------------------------------------------------------------
_______________________________________________ Gluster-users
mailing list Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
