On Thu, 10 Feb 2000, you wrote:
> >Alright, so I investigated this question a little bit more:
> >Test: 102,400,000 Megabytes to IDE disk, with Pentium 166 or something like
> >that.
> >So the timings with mmap:
> >Write: ~28 sec
> >Reading: ~20 sec
> >
> >With read/write, the speed-up stronlgly depends on how many bytes you write to
> >the disk at once:
> >
> >Write, 4 bytes (int): ~190 sec
> >Read, 4 bytes : ~ 90 sec
> >
> >Write, 1024 bytes: ~20-22 sec (this tends to be faster than mmap)
> >Read, 1024 bytes: ~20 sec
> >I experimented with a few write-lengths here, but it varies a lot because this
> >is a server, so there is not much use of giving lots of timings here. They
> >were usually 20-30 secs. And it's also hard to make sure that the data from the
> >prev read/write are not in the swap/cache. And i don't have more time to
> >experiment with this. It was interesting, but now let's speed-up the X:-).
> >It still might be a good idea to use mmap. You can't really guarantee in many
> >cases that you can read long blocks at the same time, and with mmap, you just
> >have to use it as a memory copy virtually.
> >Ferenc
>
> Make certain that you don't miss the point of mmap. It's strongest
> feature is that if you DO NOT WRITE to the datafile it saves a lot. I
> think the original question was asking about input files?
Yes it was.
>
> If you never have to write to the file mmap it as read only. By doing
> this you allow the system to only worry about paging out the data. It
> just deletes the page. If you make it read write then it needs to
> swap check the page and save out any modified data.
>
> The point that was raised about compatability is a very important
> point that is sort of being overlooked as well. Your data input files
> need to be in the correct format for the architecture you're running
> on. That is all of the floats/int must by byte aligned propperly,
> lots of RISC machines have funny requirements here. There's also the
> problem that binary data may change between different processors, best
> example is floating point.
The input format in this case is *REALLY* weird, it's almost guaranteed to
conform to no hardware on the planet. It's an ASN.1 format file. Integers
are as many bits (not bytes) as they need to be. "Real" numbers are not
denormalized IEEE but carry with with them information about the radix of the
exponent. The mantissa is subject to rules of integers. Character strings
carry information about the character set used. OTOH the ASN.1 format is the
result a a pretty cool protocal definition language that I'm researching now.
I may post a little essay to this list when I have a useful clue about it.
I any case, I'm glad to learn that I've been seriously deluded about the
performance of mmaped files.
> >
-------------------------------------------------------------- > Robert E.
Anderson email: [EMAIL PROTECTED] > Systems Programmer
phone: (603) 862-3489 > UNH Research Computing Center fax: (603) 862-1915
> -------------------------------------------------------------- > >
********************************************************** > To unsubscribe
from this list, send mail to > [EMAIL PROTECTED] with the following text in
the > *body* (*not* the subject line) of the letter: > unsubscribe gnhlug
> **********************************************************
--
email me at [EMAIL PROTECTED]
Approach love and cooking with reckless abandon.
(Isn't that a cool thought ?)
**********************************************************
To unsubscribe from this list, send mail to
[EMAIL PROTECTED] with the following text in the
*body* (*not* the subject line) of the letter:
unsubscribe gnhlug
**********************************************************
