On Jan 11, 2007, at 4:57 PM, Julian Martin Kunkel wrote:
TAS simply discards data and handles metadata efficiently in-
memory. It also
is different by using immediately completion of I/O jobs, so the
internal
handling of pvfs2 is a bit different. In the past this different view
sometimes helped the evaluation.
This sounds very interesting. I would be willing to try it out.
Ah I see so you use the states and not events, that actually was in
the
earlier versions of MPE a problem (and I think it still is).
Assume that you have one client and for example the Category
BMI_Send, you
have start and stop events, now assume that you actually see the
sequence on
one client or server (introduced by multiple parallel flows):
start, start, stop, stop
MPE state should create two overlapping states, however it doesn't,
it will
create only ONE state for actually both events, thus the resulting
logs are
wrong! This only happens if one machine (e.g. one timeline) you get
two
overlapping states !
That makes sense. This was my first time playing with MPE. I did not
read the manual like I should. I spent three days just trying to get
the logs (using libmpe_noapi) then building a Jumpshot that could
read it (the Java on my work nodes is too old to run Jumpshot and the
Jumpshot in MPICH2 does not work on my OSX).
Thus, we try to use the functions MPE_Log_get_solo_eventID and
PE_Describe_info_event to create single events and distribute them on
multiple time lines. We have a suite of slog2 transformation
programs which
allow to merge client and server logs, split overlapping actions on
multiple
timelines.
I will convert to these functions.
The question which arises is if we could synchronize the pvfs2-hint-
branch
with HEAD and will have all the stuff you need ?
If that is the case I could do so and you could use our branch
(which also
provides a patched pvfs2-cp to support the hints :),
it especially allows for MPI to show JOBS and TROVE operations,
however BMI
operations could be logged too, but not with "Request ID", but this
is not
important for the pvfs2-cp utility and could be integraded into the
log, too.
<snip>
It could be possible though to merge your client log with the
server log of
our environment. (Of course of the same run). I think this at least
will
allow you to see idle times efficiently and might help to find the
source.
That would be useful.
Our working group will provide a package with the tracing tools and
some
instructions how to use them, for you if you like, but we will need
a few
days.
However, then I have to upgrade the pvfs2-hint-branch and patch it
with all
the stuff required to run MX, if you have patches for MX against
HEAD, I
could upgrade the hint-branch to HEAD. Just tell me what you need
for MX.
I believe that HEAD has the necessary changes for bmi_mx
(Makefile.in, configure.in, changes to BMI, etc.) except for the
files in src/io/bmi/bmi_mx (mx.h, mx.c, module.mk.in).
If you need mx.[ch], let me know. I do not want to commit them yet
until I have done a little more testing.
Scott
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
