Hi Randy,
It doesn't look like they actually need to use MAP_SHARED there,
because they never modify the mapped memory region (they open the file
with O_RDONLY, and only pass PROT_READ to mmap). If they were
modifying the file elsewhere and then expecting the updates to be seen
in the mapped memory, then they would need MAP_SHARED, but it doesn't
look like they do that either. Could you change that MAP_SHARED to
MAP_PRIVATE, recompile, and run everything on PVFS again? I think we
can probably send the blast folks a patch for that, but we want to
make sure that it works first.
Thanks,
-sam
On Jun 16, 2009, at 8:10 AM, Randall Martin wrote:
We download the source from http://www.mpiblast.org/Downloads/
Stable. The
mmap() call with MAP_SHARED is in mpiblast-1.5.0-pio/ncbi/corelib/
ncbimem.c.
-Randy
From: Becky Ligon <[email protected]>
Reply-To: Becky Ligon <[email protected]>
Date: Mon, 15 Jun 2009 17:49:46 -0400 (EDT)
To: Sam Lang <[email protected]>
Cc: Becky Ligon <[email protected]>, Phil Carns
<[email protected]>, Randall
Martin <[email protected]>, <[email protected]>
Subject: Re: [Pvfs2-users] help with pvfs2 and formatdb (used with
BLAST)
Sam:
Randy will have to answer that question for you, since he is the
one who
actually ran the command. I'll get with him in the morning and
find out.
Becky
Hi Becky,
Does clemson build formatdb (and blast) from the source code tarball
ncbi? The reason I ask is that we might be able to make a small
modification (switching from MAP_SHARED to MAP_PRIVATE) to allow
blast
to work on PVFS deployments. I can give further info if that's
something you all are willing to try.
Thanks,
-sam
On Jun 11, 2009, at 9:18 AM, Becky Ligon wrote:
Phil:
We have a 4Gb input file for the formatdb command. The problem
doesn't
seem to occur until after the command has crunched for several
hours.
I'll be happy to share the file with you (and the formatdb command
that
Randy is using). Just let me know the best way to get the file to
you...I
can ftp it to you or put it on a machine that you still have access
to....just let me know!
Becky
Hi Becky,
Is there an example input file (either for public download or that
you
could send to us) that triggers the problem? I tried this trivial
example:
http://en.wikipedia.org/wiki/Fasta_format#Header_line
... by just pasting it into a text file and running "formatdb -i
<file>". I also tried just concatenating the same input together
many
times to get a bigger file, but I didn't have any luck
triggering the
problem.
-Phil
Sam Lang wrote:
Hi Becky,
pvfs2_file_write is a kernel module function, so I think that
message
must have come from the kernel log. There may not be any
messages
in
the client log (/tmp/pvfs2-client.log), but I was hoping there
would be
something that might point us at the problem (the client daemon
got
restarted, for example). Also, the server logs might have
something.
As a last resort, you could try to enable debugging in the kernel
module
to see what happens around the time of another error.
-sam
On Jun 9, 2009, at 3:19 PM, Becky Ligon wrote:
The second message (pvfs2_file_write) came out of the client log
(/tmp/pvfs2-client.log).
We are thinking that either the client or server processes are
running
out
of system resources.....maybe file descriptors, for example.
We have now put PVFS on our Palmetto cluster as the primary
parallel
file
system .... no more lustre! So, now we are trying to iron
some of
these
problems, which is why I think the problem is really system-
resource
related.
I found an old thread from Dec of 2007 where a guy in Beijing
had
the
same
problem; unfortunately, there was no response to his problem.
Becky
Hi Becky,
What about the client log? Anything in there?
-sam
On Jun 9, 2009, at 10:42 AM, Becky Ligon wrote:
PVFS-USERS:
When I use formatdb to create a BLAST-usable database, I get
the
error:
[formatdb] ERROR: Failed to create index: ISAMErrorCode -5.
The PVFS kernel log shows the following error:
pvfs2_file_write: error in vectored write to handle xxxx,
FILE:
xxxx.
(the x's have values but are just specific to my database).
I saw a thread in the pvfs2-users archive where someone else
has
this same
problem, but there were no responses to it. Am I just
having a
setup
issue with PVFS?
We are running PVFS 2.8.
Any help would be apprecited.
Thanks,
Becky Ligon
--
Becky Ligon
Clemson University
864-656-3865
_______________________________________________
--
Becky Ligon
PVFS Developer
Clemson University
864-656-3865
--
Becky Ligon
PVFS Developer
Clemson University
864-656-3865
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
