Sam,
In the <StorageHints> context:
TroveMethod alt-aio
Ah! thanks!
Unless the data he's just written is sitting in the kernel buffers, I
would expect reads to have the same problem as writes if aio is the
cause. What makes you suspect AIO libraries for his platform?
Oh I didn't realize his reads were better.. I just jumped to a conclusion
because Florin gave me access to his cluster machine and when I set it up
I ran configure with --disable-aio-threaded callbacks since without that
his pvfs2 setup just sat there and did nothing..:)
No I/Os were being completed.
If it is the same cluster that he is talking about here then I assumed it was
most likely due to that.
Do you mean configure is disabling threaded callbacks for his build,
or that we should ask it to? AIO results we've seen without threaded
callbacks are worse than with them.
Without disabling his setup does not even work. It is a ppc64 based
cluster with a fairly ancient glibc if I am not mistaken.
yeah.. thats what I thought too..
:-) I hear you. Are you running over ext3 Murali? I've seen
results that suggest xfs might be better for large IOs and multiple
threads.
On my home machine, yes.
On my latop, no. I run over NTFS eventually since my "virtual disk
files" are hosted off NTFS ;)
XFS rocks for such workloads indeed.
thanks,
Murali
-sam
> Thanks,
> Murali
>
> On 7/17/07, Florin Isaila <[EMAIL PROTECTED]> wrote:
>> Hi Sam, we start the pvfs2 servers on different machines than the
>> compute nodes (picking the nodes from the list provided by the batch
>> system). Was that your question?
>>
>> And I should have said, all the measurements are done with collective
>> I/O of ROMIO.
>>
>> On 7/17/07, Sam Lang <[EMAIL PROTECTED]> wrote:
>> >
>> > Ah, I read your email wrong. Hmm...so writes really tank. Are you
>> > using the storage nodes as servers, or other compute nodes?
>> >
>> > -sam
>> >
>> > On Jul 17, 2007, at 11:15 AM, Sam Lang wrote:
>> >
>> > >
>> > > Hi Florin,
>> > >
>> > > Just one clarification question...are those are bandwidth numbers
>> > > not seconds as the plot label suggests?
>> > >
>> > > -sam
>> > >
>> > > On Jul 17, 2007, at 11:03 AM, Florin Isaila wrote:
>> > >
>> > >> Hi everybody,
>> > >>
>> > >> I have a question about the PVFS2 write performance.
>> > >>
>> > >> We did some measurements with BTIO over PVFS2 on lonestar at
>> TACC
>> > >> (http://www.tacc.utexas.edu/services/userguides/lonestar/)
>> > >>
>> > >> and we get pretty bad write results with classes B and C:
>> > >>
>> > >> http://www.arcos.inf.uc3m.es/~florin/btio.htm
>> > >>
>> > >> We used 16 I/O servers, the default configuration parameters
>> and upto
>> > >> 100 processes. We realized that all I/O servers were used
>> also as
>> > >> metadata servers, but BTIO uses just one file.
>> > >>
>> > >> The times are in seconds, contain only I/O time (no compute
>> time) and
>> > >> are aggregated per each BTIO run (BTIO performs several writes).
>> > >>
>> > >> TroveSyncMeta was set to yes (by default). Could this cause
>> the I/
>> > >> O to
>> > >> be serialized? It looks as if there were a serialization.
>> > >>
>> > >> Or could the fact that all nodes were also launched as metadata
>> > >> managers affect the performance?
>> > >>
>> > >> Any clue why this happens?
>> > >>
>> > >> Many thanks
>> > >> Florin
>> > >> _______________________________________________
>> > >> Pvfs2-users mailing list
>> > >> [email protected]
>> > >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>> > >>
>> > >
>> >
>> >
>> _______________________________________________
>> Pvfs2-users mailing list
>> [email protected]
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>
>
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
