For the clients, cat /proc/fs/lustre/version For the servers, it¹s the same, but presumably you don¹t have access.
On 5/19/15, 11:01 AM, "Schneider, David A." <[email protected]> wrote: >Hi, > >My first test was just to do the for loop where I allocate a 4MB buffer, >initialize it, and delete it. That program ran at about 6GB/sec. Once I >write to a file, I drop down to 370mb/sec. Our top performance for I/O to >one file has been about 400 mb/sec. > >For this question: Which versions are you using in servers and clients? >I don't know what command to determine this, I suspect it is older since >we are on red hat 5. I will ask. > >best, > >David Schneider >________________________________________ >From: lustre-discuss [[email protected]] on behalf >of John Bauer [[email protected]] >Sent: Tuesday, May 19, 2015 8:52 AM >To: [email protected] >Subject: Re: [lustre-discuss] problem getting high performance output to >single file > >David > >You note that you write a 6GB file. I suspect that your Linux systems >have significantly more memory than 6GB meaning your file will end being >cached in the system buffers. It wont matter how many OSTs you use as >you probably are not measuring the speed to the OST's, but rather, you >are measuring the memory copy speed. >What transfer rate are you seeing? > >John > >On 5/19/2015 10:40 AM, Schneider, David A. wrote: >> I am trying to get good performance with parallel writing to one file >>through MPI. Our cluster has high performance when I write to separate >>files, but when I use one file - I see very little performance increase. >> >> As I understand, our cluster defaults to use one OST per file. There >>are many OST's though, which is how we get good performance when writing >>to multiple files. I have been using the command >> >> lfs setstripe >> >> to change the stripe count and block size. I can see that this works, >>when I do lfs getstripe, I see the output file is striped, but I'm >>getting very little I/O performance when I create the striped file. >> >> When working from hdf5 and mpi, I have seen a number of references >>about tuning parameters, I haven't dug into this yet. I first want to >>make sure lustre has the high output performance at a basic level. I >>tried to write a C program uses simple POSIX calls (open and looping >>over writes) but I don't see much increase in performance (I've tried 8 >>and 19 OST's, 1MB and 4MB chunks, I write a 6GB file). >> >> Does anyone know if this should work? What is the simplest C program I >>could write to see an increase in output performance after I stripe? Do >>I need separate processes/threads with separate file handles? I am on >>linux red hat 5. I'm not sure what version of lustre this is. I have >>skimmed through a 450 page pdf of lustre documentation, I saw references >>to destructive testing one does in the beginning, but I'm not sure what >>I can do now. I think this is the first work we've done to get high >>performance when writing a single file, so I'm worried there is >>something buried in the lustre configuration that needs to be changed. I >>can run /usr/sbin/lcntl, maybe there are certain parameters I should >>check? >> >> best, >> >> David Schneider >> _______________________________________________ >> lustre-discuss mailing list >> [email protected] >> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > >-- >I/O Doctors, LLC >507-766-0378 >[email protected] > >_______________________________________________ >lustre-discuss mailing list >[email protected] >http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org >_______________________________________________ >lustre-discuss mailing list >[email protected] >http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
