Thanks Dean. Well, my /etc/exports file has this entry:
/export 10.0.0.0/255.0.0.0(rw,async)
and the pvfs2-fs.conf file has these entries:
<StorageHints>
TroveSyncMeta Yes
TroveSyncData No
</StorageHints>
So both nfs and pvfs2 are set to async. But the meta is set to sync, it may
be the reason for this. Hmm... Still not clear! Maybe I need to set Meta to
'no' even though its not advisable.
Thanks
nasr
From: Dean Hildebrand <[EMAIL PROTECTED]>
To: "Nasr Y.M.J.O." <[EMAIL PROTECTED]>
CC: [EMAIL PROTECTED], [email protected]
Subject: Re: [Pvfs2-users] PVFS2 Performance
Date: Wed, 20 Sep 2006 13:03:29 -0400
NFS:
-----
Access size per process = 67108864 bytes, ntimes = 5
Write bandwidth without file sync = 70.134351 Mbytes/sec
Write bandwidth including file sync = 71.789752 Mbytes/sec
PVFS2
--------
Access size per process = 67108864 bytes, ntimes = 5
Write bandwidth without file sync = 329.618503 Mbytes/sec
Write bandwidth including file sync = 121.779536 Mbytes/sec
I did the test after tuning TCP using netpipe benchmark. I can see there
is a big gap between write and read with sync in pvfs2 as compared to NFS
(121, 320 compared to 71, 80). Why?
I'll take a stab at this. In order for NFS to maintain open-to-close
semantics, all data is flushed to stable storage at file close. Issuing a
fsync just before file close does not do anything extra. So your NFS
experiments are always doing the same thing. I would guess PVFS2 does not
sync data to stable storage unless it is specifically requested.
If you want to try removing this behavior from NFS, simply change your
/etc/exports file to export data asynchronously. Once this is done, your
server may or may not flush data to stable storage. Here is the relevant
part from the exports man page:
async This option allows the NFS server to violate the NFS
protocol
and reply to requests before any changes made by that
request
have been committed to stable storage (e.g. disc drive).
Using this option usually improves performance, but at the
cost
that an unclean server restart (i.e. a crash) can cause data
to
be lost or corrupted.
In releases of nfs-utils upto and including 1.0.0, this
option
was the default. In this and future releases, sync is
the
default, and async must be explicit requested if needed.
To
help make system adminstrators aware of this change,
âexportfsâ
will issue a warning if neither sync nor async is specified.
Dean
Many thanks,
nasr
---------------------------------------------------------------------------------------------------------------------------------------
io.c program error
# /opt/mpich1/intel/bin/mpicc io.c -g -Wall -o io
io.c(49): error: identifier "O_LARGEFILE" is undefined
int amode = O_RDWR | O_CREAT | O_LARGEFILE;
^
io.c(53): remark #1419: external declaration in primary source file
int parse_args(int argc, char **argv);
^
io.c(54): remark #1419: external declaration in primary source file
double Wtime(void);
^
io.c(57): remark #1419: external declaration in primary source file
extern char *optarg;
^
io.c(58): remark #1419: external declaration in primary source file
extern int optind, opterr;
^
io.c(58): remark #1419: external declaration in primary source file
extern int optind, opterr;
^
compilation aborted for io.c (code 2)
-------------------------------------------------------------------------------------------------------------------------------------------
_________________________________________________________________
Find love online with MSN Personals.
http://match.msn.com.my/match/mt.cfm?pg=channel
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
--
Dean Hildebrand
Ph.D. Candidate
University of Michigan
_________________________________________________________________
Find love online with MSN Personals.
http://match.msn.com.my/match/mt.cfm?pg=channel
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
