Hi Murali, Our test environment are: windows clients and linux SMB Server/FTP server with PVFS2 client and PVFS Server with JBOD storage,XFS filesystem connect with GE Switch
I do some tests,and the results are : 1.Copy 2G files from windows clients to SMB Server(PVFS2 Client), 20MB/S. We print the IO request size from "pvfs2_file_write", the count=61440 2.Copy 2G files form windows clients to FTP server (PVFS2 Client), 50MB/S. We print the IO request size form "pvfs2_file_write", the count=20480 3.Copy 2G files form PVFS2 client local disk to the pvfs2-fs, 110MB/S. We print the IO request size from "pvfs2_file_write", the count=4194304, It's so good! The SMB and the FTP use the "write" function but not the "writev" function. My questions are: 1.SMB IO request size is bigger than FTP IO request size, in theory the SMB should be faster than FTP,but actually the fact is the SMB is much slower than the FTP. Why? 2.If use the some IO request size, "writev" function, "write" function, "aio_write" function,which is best? 3.I know the PVFS2 cut the files with 64kB, I want to know the IO request size between the PVFS2 client and PVFS2 server.How can I get the size? Thank you very much! I expect your letter! Best regards ---------------------------------------------- Jiang Yi Huawei Technologies Co.,Ltd. Application Platform Solution Department Tel: (86-755) 28567481 Email: [EMAIL PROTECTED] ----- Original Message ----- From: <[EMAIL PROTECTED]> To: "??" <[EMAIL PROTECTED]> Cc: <[email protected]> Sent: Tuesday, October 03, 2006 3:28 PM Subject: Re: [Pvfs2-users] Samba over PVFS2 performance > Hi Jiang, > > > HI Murali, > > I test the write performance just on pvfs2 client with big blocks,the > > performance seems good. > > and I test samba with COPY form windows clients to local disk(JBOD) > > without pvfs2,the local file system is XFS,and the write performance is > > good too.about 100MB/s. > > But when I add the pvfs2 file system,the write performance is bad,just > > 20MB/s. > > Hmm.. This definitely seems to indicate that Samba over PVFS2 uses very > small block sizes.. > > > 1.you said " Due to the absence of any client-side caches, multiple round > > trips ensues and performance drops.",shall I use the client-sid caches?How > > can I use it ? > > Nope. We haven't implemented this yet. > > > 2.you said "Can you check if the smbd server uses pvfs2 writev function > > instead of write? (perhaps by strace'ing the server?)", > > How can I check it ? > > If the samba server is a UNIX userspace process, could you start up the > samba server like so > strace -f -o /tmp/log.txt -e file /path/to/smbd <arguments> > AFter the server is started, remount samba on all clients and redo your > tests. > > Check the contents of the file /tmp/log.txt and see if there are calls to > writev or write()?If there are calls to write(), check the sizes of the > system call arguments.. > Hopefully that will point us to the problems and possibly solutions. > THanks, > Murali > > > Thank for help me very much! > > > > Best regards > > ---------------------------------------------- > > Jiang Yi > > Huawei Technologies Co.,Ltd. > > Application Platform Solution Department > > Tel: (86-755) 28567481 > > Email: [EMAIL PROTECTED] > > > > ----- Original Message ----- > > From: "Murali Vilayannur" <[EMAIL PROTECTED]> > > To: "jiangyi 60012480" <[EMAIL PROTECTED]> > > Cc: <[email protected]> > > Sent: Monday, September 25, 2006 11:43 PM > > Subject: Re: [Pvfs2-users] Samba over PVFS2 performance > > > > > >> Hi Philips, > >> > >> > I met a Samba over PVFS2 prolem.So would anybody help me please? > >> > I use the pvfs2 file system and export the files with Samba.The > >> hardware are two JBOD with > >> > 146G 10k FC disk, 4 pvfs2 IO servers,8 pvfs2 clients and some > >> windows clients.Every IO server > >> > manages 6+1 RAID5.the stripe size is 64k. > >> > My problem is that How can we tuning the read/write performance of > >> this system ? > >> > this architecture,I test the performance with 2G files.and find > >> the read performance is fast,but the > >> > write performance is bad.Would you tell me what should be care > >> about Samba over PVFS2? > >> > >> Can you verify whether PVFS2 performance (no Samba export, just native > >> PVFS2) is bad on your setup first? What > >> benchmark did you use to measure the performance on your system? > >> Just wanted to make sure that it is a pvfs2 performance problem and not > >> a > >> samba tuning issue.. > >> > >> I suspect what might happen with writes is that the VFS/smbd writes out > >> single pages and hence PVFS2 sees only 4K/8K bytes at a time. > >> Due to the absence of any client-side caches, multiple round trips > >> ensues > >> and performance drops. > >> Can you check if the smbd server uses pvfs2 writev function instead of > >> write? (perhaps by strace'ing the server?) > >> thanks, > >> Murali > >> > > > > > > > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
