hy ,
i get much higher rates between AFS Server and my Clients :
[EMAIL PROTECTED] root]# bonnie++ -d /afs/test11a/ -s 2008
Using uid:0, gid:0.
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
xxx21.de 2008M 9584 53 13950 33 3404 10 4685 18 5843 4 903.6 7
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 251 11 2401 25 480 11 345 15 1472 26 564 10
xxx21.de,2008M,9584,53,13950,33,3404,10,4685,18,5843,4,903.6,7,16,251,11,2401,25,480,11,345,15,1472,26,564,10
i have identically results from 2 Clients running bonnie at the same time to 1 Server . i only changed the cache to memache , without any other changes to the clients or server .
the wired thing for me is , my Server is not going over 25% CPU Utilization when i run this test . and also the Clients are not busy .
the clients are connected with Gigabyte Ethernet (Fiber) to the Server on a local switch , without any other System (it is a test environment) .
The Server is Connected to an IBM Shark with 2 Gigabit Fiberchannel . when i cat a file locally on the Server to dev/null from /vicepa i get rates about ~70 Mb/s .
So where so i loos so many Bandwidth ??
i use Openafs Server 1.2.8 on an AIX 4.3.3 System and the Clients are Linux Redhat 8.0 also with Openafs 1.2.8.
a copy between the Systems itself with ttcp i get network rates >100 Mb/s so Network is also not the problem ...
Sven
| Nathan Ward <[EMAIL PROTECTED]>
Sent by: [EMAIL PROTECTED] 02/28/2003 02:40 PM |
To: [EMAIL PROTECTED] cc: Michael Robokoff <[EMAIL PROTECTED]>, openafs <[EMAIL PROTECTED]> Subject: Re: [OpenAFS] Performance |
This is for in-house developed stuff for black box network appliances,
we don't spend any time in configure, and about 20% in install.
I'll try building our entire tree in afs with 2000000 cache size and
-chunksize 18 a bit later on and let you know.
Nathan Ward
[EMAIL PROTECTED] wrote:
> I recently did some compile test comparisons, and the numbers I got
> were:
>
> Local Disk NFS AFS
> 4:23 4:48 4:31 minutes
>
> This seems reasonable, as the actual compilation itself is >90% of
> the time (including running configure and install).
>
> A 4x slowdown does seem unreasonable, unless your AFS server is using
> a 2400 baud modem for its network ;-). Or unless your cache is so
> small that the files you create are always exceeding the cache size.
> ------------------------------------------------------------------------
> --
> Edward Moy
> Apple Computer, Inc.
> [EMAIL PROTECTED]
>
> (This message is from me as a reader of this list, and not a statement
> from Apple.)
>
> On Thursday, February 27, 2003, at 03:41 PM, Nathan Ward wrote:
>
>> I'll do some tests with this today.
>>
>> Yes of course it will be slower, but it is unbearably slower. I'm
>> not sure about local disk stats, but perhaps a better comparison:
>> It takes ~4 times as long on AFS over NFS, last I checked.
>>
>> [EMAIL PROTECTED] wrote:
>>
>>> Sorry for any confusion, but it was a 1 GB file I used. So
>>> performance was not up to the theoretical maximum, but I was not
>>> on a quiet network, and the server was on a different subnet.
>>> That is why one has to compare the performance with NFS directly,
>>> and not against the theoretical maximum (the server in the test
>>> was doing both NFS and AFS).
>>>
>>> Other than setting the cache size to 2000000 blocks, I used
>>> "-chucksize 18" to get the 256 KB chucksize. NFS write time was
>>> about 4:20 compared to 3:30 for 256 KB chucksize and 4:50 for the
>>> default 64 KB chucksize. System CPU time for NFS was about the
>>> same as AFS (around 20 seconds).
>>>
>>> As for compiling on AFS versus local disk, of course AFS would be
>>> slower. The files have to be read from the network at least once,
>>> and new/modified files written back. However, compiling on NFS,
>>> for example, will generally be poorer, though perhaps not
>>> strikingly so, depending on how much your kernel caches NFS. And
>>> as more users access files simultaneously, the performance of NFS
>>> will degrade faster than AFS.
>>>
>>> On Thursday, February 27, 2003, at 01:24 PM, Nathan Ward wrote:
>>>
>>>> Yes, playing with tuning the cache parameters is something thats
>>>> on my run queue.
>>>> What were your afs client settings you had when you did the 2gb
>>>> file in 3.5 min?
>>>>
>>>> My calculations say that 2gb in 3.5 minutes is ~10mb/s. A 100Mbit
>>>> network can do ~12.5MB/s of packets, so allow for headers and
>>>> control data and i'd say your are limited at this point by your
>>>> network.
>>>>
>>>> You say it beats NFS, whats the CPU usages for 2GB over OpenAFS
>>>> vs NFS? Context switches?
>>>>
>>>> My developers complain that code compiles are much much slower in
>>>> AFS than on the local disk, but actual file performance when they
>>>> aren't compiling is not-so-bad. I imagine this is because of the
>>>> CPU that AFS takes to get the files, so CPU usage is important >>>
>>>> here.
>>>>
>>>> [EMAIL PROTECTED] wrote:
>>>>
>>>>> As I understand it, the cache helps reads primarily, though if
>>>>> you are doing random updates of a file, those also get cached,
>>>>> until you do the finally close, which then writes all to the
>>>>> server. That assumes the file fits in the cache. If not, then
>>>>> a write can force the cache to partially empty, so the latest
>>>>> write can be cached.
>>>>>
>>>>> But there seems to be some pathological behavior, because doing
>>>>> a sequential write should ideally not be affected that much by
>>>>> size of the cache. But in a test I did with a 1 GB file, with
>>>>> regular network traffic, it took almost 5 minute to write with
>>>>> with a 2 GB cache (about 10% slower than NFS in this case), but
>>>>> took almost 11 minutes with a 0.5 GB cache. The system CPU
>>>>> time rose from 20 seconds for the 2 GB cache to 517 seconds
>>>>> with the 0.5 GB cache.
>>>>>
>>>>> As for being slower than NFS in this test case, with such a
>>>>> large cache, tuning the cache parameters does help. Setting
>>>>> the chucksize to 256KB dropped the 2 GB cache speed to about
>>>>> 3.5 minutes, beating NFS.
>>>>>
>>>>> On Thursday, February 27, 2003, at 11:51 AM, Nathan Ward wrote:
>>>>>
>>>>>> Well I'd expect that it goes slower as your cache size is
>>>>>> exceeded as it then needs to start getting that data to the
>>>>>> server. Or is the cache for read operations only?
>>>>>>
>>>>>> I notice that there are around about the same number of
>>>>>> packets/sec as context switches/sec on my client machines. I
>>>>>> wonder if switches between userland and kernel could be to
>>>>>> blame... ? Who sends packets in OpenAFS, the userspace daemon
>>>>>> or the kernel?
>>>>>>
>>>>>> Nathan
>>>>>>
>>>>>> [EMAIL PROTECTED] wrote:
>>>>>>
>>>>>>> Could the slowness you see with your dd write test be related
>>>>>>> to the cache exhaustion issue that I raised recently, when
>>>>>>> writing a file larger than your cache size. Your test
>>>>>>> writes a 1 GB file, so if your cache is smaller than this,
>>>>>>> you will see poor performance once your cache size is exceeded.
>>>>>>>
>>>>>>> On Thursday, February 27, 2003, at 11:15 AM, Nathan Ward wrote:
>>>>>>>
>>>>>>>> I see pretty bad performance to tell you the truth.
>>>>>>>> I can read and write ~60mb/s directly to my raid array, but
>>>>>>>> when using OpenAFS (locally or remotely) to the same array,
>>>>>>>> I get around 6-10MB/s, I have seen up to 25MB/s over a
>>>>>>>> peice of 1000Mbps fibre. Client and Server are both dual
>>>>>>>> P3-1ghz with 1024mb ram. I notice the context switches on
>>>>>>>> the server at this time jump to ~10000/s, and on the client
>>>>>>>> ~40000/s. I imagine this is the source of my slowdown, but
>>>>>>>> I havn't had a chance to look into it.
>>>>>>>>
>>>>>>>> I'd be interested if anyone else has the same level of
>>>>>>>> context switches going on.
>>>>>>>>
>>>>>>>> This is while doing a large sequential write operation (dd
>>>>>>>> if=/dev/zero of=/afs/alb-nz.esphion.com/public/dd.out
>>>>>>>> bs=256k count=4096).
>>>>>>>>
>>>>>>>> Michael Robokoff wrote:
>>>>>>>>
>>>>>>>>> Does anyone have any Open AFS performance information they
>>>>>>>>> can share with me. I plan on doing a couple benchmarks and
>>>>>>>>> I would like to have some idea of what to expect.
>>>>>>>>
>
>
>
_______________________________________________
OpenAFS-info mailing list
[EMAIL PROTECTED]
https://lists.openafs.org/mailman/listinfo/openafs-info
