On Feb 21, 2008, at 1:58 PM, Phil Carns wrote:
Sam Lang wrote:
On Feb 14, 2008, at 4:34 PM, Pete Wyckoff wrote:
Add --enable option to build a threaded pvfs2-client-core, that is,
an executable linked against pvfs2-threaded.so.
Remove the "--threaded" argument from pvfs2-client. Whatever
version
of client-core that was configured is all that is available at
runtime.
It will always be installed as pvfs2-client-core.
---
Makefile.in | 15 +++++++--------
configure | 17 +++++++++++++++--
configure.in | 20 ++++++++++++++++++++
src/apps/kernel/linux/module.mk.in | 9 ++++++---
src/apps/kernel/linux/pvfs2-client.c | 31 +
+-----------------------------
5 files changed, 50 insertions(+), 42 deletions(-)
What does everybody think about this? I had complained a long time
ago about always getting a pvfs2-client-core and
pvfs2-client-core-threaded during a kernapps build. The latter
brings in the entire libpvfs2-threaded.a and all the objects for
that.
I don't have a feel for who uses the threaded client core. The only
way to use it now is to start pvfs2-client with the "--threaded"
option. So I made the --enable option default to off. Is there a
reason to want both threaded and non-threaded versions installed?
I think the patch looks great. My only concern is that this will
make the threaded client daemon never get used. My view is that we
should probably have the threaded version as the default. I think
in the general case, the threaded version is going to perform
better. Its pretty hard to buy a system that doesn't have at least
two cores today, and even on a uni-processor system, the
performance of the threaded daemon shouldn't be much worse than the
non-threaded.
I originally added the threaded client functionality to see if it
would help performance, but it turned out to be buggy. Maybe with
Phil's recent thread-safety fixes to the system interfaces, we
should revisit making the threaded daemon the default.
FYI, I think the threaded client just got a boost in stability.
There was a subtle bug in the how the job thread mgr was dealing
with the device that made it possible for it to pull operations out
that could not be handled. I think this could technically happen
even with the non-threaded client, but this hasn't been a big deal
because it could only be triggered with a) a lot of concurrent
operations b) and a pvfs2-client-core daemon running fast enough to
empty out the device queue (much easier w/ a dedicated thread).
There was also in the last week or two a fix to how bmi_tcp kept
track of socket errors that I suspect was relevant to the threaded
client core as well, but I was tracking that down in the context of
some server stuff so I can't say for sure.
I agree with everything in this email thread, just commenting on the
state of the threaded client in case anyone following was curious.
I'm not sure that the overhead of a single operation will be
affected much, honestly. We can't disable threads and locking
entirely, since we need the remount thread running. I think the
patch I just committed to fix the segfault in HEAD will likely help
with individual operation performance more than disabling threads
would. Just my 2c though.
Its hard to get a feeling for how much the threaded daemon would
help without a thorough performance analysis of the two on a number
of different systems. Something I would like to do at some point,
but its pretty low on the list.
I don't have a good feel for the threaded / non threaded impacts
performance either without looking into it more.
For the non-threaded version, that remount thread really bugs me.
It seems like there has to be another way to accomplish whatever its
doing. Its a shame to have to pull in lpthread and turn on all of
the library locks when the client-core doesn't need either for 99%
of what it does.
Yeah I agree. I'm pretty sure it could be done with another state
machine like our unexpected_sm.
-sam
-Phil
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
