Hi Henk,
Please be sure to CC pvfs2-users on future emails.
Without any additional information, my guess is that every application
you're using in this workflow performs very small I/Os. These
operations are passed into the kernel, back out to pvfs2-client,
across the network and received by the PVFS server, who then performs
I/O on your application's behalf. If operations are particularly
small, this can be a lot of overhead.
Other networked file systems can hide some of this latency by caching
data (either coherently or not) on the client. PVFS does not do this,
so each little operation goes across the wire. There's really no
advantage to using a parallel file system for the workload you have
described, unless you're planning on having a lot of systems doing
this process in parallel and want a single place to store the output.
What sort of network do you have in this system? What sort of nodes
are you using for the PVFS servers?
Regards,
Rob
On May 13, 2008, at 3:21 PM, belcampo wrote:
Rob Ross wrote:
Hi Henk,
Can you provide more information on the system that you're using
(Linux version, # of nodes, type of nodes, type of storage, rate of
I/O to local storage, network type, units on the below measurements)?
Thanks,
Rob
On May 13, 2008, at 12:26 PM, belcampo wrote:
Hi all,
Have 2.7.1 installed and see things I don't understand, maybe
someone could shed some light ?
Copying dvd.iso with cp -r from pvfs2 to local takes 1:34.79
I'm absolutely happy with this, 64MB/sec; hdparm gives 73MB max
Local extracting of audio and video takes 2:47.63
I'm absolutely happy with this
From pvfs2 the same action takes approx 2:40:00
After 8 minutes 5% is done.
It was actually 2:06:07
I don't understand why this takes
2:06:07 vs 2:47.63 = 7567 / 126 = 60 times as long
Henk Schoneveld
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
