On Mar 8, 2011, at 8:56 PM, Milo Velimirović wrote: > On Mar 8, 2011, at 5:08 PM, William H. Magill wrote: >> On Mar 8, 2011, at 8:25 AM, Milo Velimirović wrote: >>> On Mar 7, 2011, at 11:03 PM, Chris Murphy wrote: >>>> On Mar 7, 2011, at 8:25 PM, Milo Velimirović wrote: >>>>> On Mar 7, 2011, at 7:06 PM, Chris Murphy wrote: >>>>>> On Mar 7, 2011, at 6:02 PM, Markus Hitter wrote: >>>>>>> Am 07.03.2011 um 20:22 schrieb Chris Murphy: >>>>>>> >>>>>>>> I don't know the difference with XNU between block level and raw device >>>>>>> >>>>>>> For reading the whole thing, the result should be the same for both. A >>>>>>> "cmp", "diff" or two "md5" should tell you wether I'm speaking the >>>>>>> truth ;-) >>>>>> >>>>>> I agree. Should be the same for both with respect to performance as >>>>>> well, but they aren't. 6x slower for block level compared to raw. >>>>> >>>>> Yes, the resulting files should be the same; no, the copy should not be >>>>> the same speed! It's hard for me to believe that this confusion about >>>>> block vs. character/raw devices still exists when UNIX is more than 40 >>>>> years old. >>>> >>>> OK well, then it should be simple for you to explain why /dev/disk0 >>>> results in 17MB/s read, and /dev/rdisk0 results in 107MB/s read, and on >>>> Linux /dev/sda result in 107MB/s read which is the block level device and >>>> there is no raw device created anymore. >>> >>> I can't speak for Linux, though it would appear based on your performance >>> numbers that the block and raw devices were unified using the raw model. >>> Historically, in the UNIX world, block devices had I/O done (surprise!) >>> block at a time into a set of kernel maintained buffers and data was copied >>> to/from the disk buffer cache from/to user space. Raw access bypassed this >>> disk cacheing/buffer mechanism. >> >> In the Unix (tm) world, for a very long time, block I/O was limited to 256 >> bytes at a time -- as some very deep Kernel I/O code only had 256 byte >> buffers. > > I don't believe it was ever 256 byte buffers. pdp11 UNIX and descendants used > 512 byte disk blocks and buffers. Consult Lions' commentary or the source > code itself. 256 words == 512 bytes in the pdp11.
You are most likely right. (All this happened a long time ago, and there is not a lot of documentation still easily available since HP shut down ftp.dec.com) However, 256 or 512 bytes is tiny when you are talking about tracks that hold 4/8/16 times that much data. You can only read what passes by the head, and if you have to wait for multiple revolutions to read a single track your latency goes up dramatically. >> There were quite a few hands rung in the beginning of OSF/1. People could >> not figure out why I/O was so slow on the new high capacity (at the time) >> disk drives. >> >> Why? Because it took four 256 byte reads to read a single 1024 byte track. >> It took a hardware engineer hand-wiring scope probes into a box to discover >> that the "instrumentation" built into the Kernel was simply measuring all >> the wrong things and at much too high a level.... and the I/O code was >> discovered to have been one of the few pieces of code that nobody looked at >> ... simply because it was so well written that it "worked" all the time with >> anything. :) > > This may or may not be true with allowances for block size and tracks. It > seems like an apocryphal story because it occurs too late in the timeline of > the history of UNIX or it may just be about some other UNIX than OSF/1. One > should really look at the work done by Marshall Kirk McKusick et al, to > address the problem of I/O performance. This was done when he was at CSRG at > Berkeley and resulted in the Berkeley Fast File System. The FFS work was > published in the early to mid 1980s and OSF/1 didn't appear until after 1988 > when computer manufacturers came together to form the Open Software > Foundation -- the OSF. To suggest that nobody looked at a section of UNIX > code because it worked so well seems odd to me. Links to the papers > describing the Fast File System are included in this wikipedia page, > http://en.wikipedia.org/wiki/Unix_File_System The I/O block size problem was found and fixed by DEC. DEC, at that point in time was selling Ultrix on Vaxen and had introduced the MIPS based DECstation (Risc/Ultrix). Ultrix was pretty pure BSD 4.2 and later 4.3 (Tahoe and Reno)BSD Unix. Marshal's Fast File system was always part of Ultrix. (As I recall, DEC never really supported BSD 4.4, but switched to OSF/1 at that time - 1988. (I may have the dates off. All this happened long ago.) Ultrix worked quite well until DEC began to dramatically expand the capacities of their hard drives as they moved from the VAX "mainframe" architecture to MIPS "desktop" as part of the Workstation Consortium. [As an aside, It seems that BSD was not as "architecture independent" as the authors had "assumed." Amongst other things, when BSD was ported to MIPS from VAX (i.e. PDP) architecture and later to the Power 6/32 platform, a huge portion -- I was told over 60% -- of the BSD code had to be"fixed", i.e. rewritten because of architecture (pdp/11) dependencies.] At any rate, that is when they discovered that they could dramatically improve the I/O performance of Ultrix/OSF/1 by re-writing the I/O modules to do much larger physical reads than were presently being done by the FSF. Cutting the number of I/Os necessary to read a single track of data from a disk drive was a big win. I was told by the folks in DEC Engineering at the time that it was a simple fix to expand the buffer sizes, but one that had simply been overlooked because the code worked.... part of the motto... "if it ain't broke, don't fix it." The issue and the fix were part of the old hardware dependency issues that surrounded the historical development of BSD. Remember, the RP04 was only a 92MB drive (36bit words) with a 5.6 microsecond/Word transfer rate circa mid 1970s ... a washing machine with removable disk pack with about 20 10 inch diameter platters! The MIPS workstations lead DEC"s transition into 5-1/8 form factor "modern" disk drives, accessed either via TurboChannel or SCSI. OSF/1 never "officially" ran on the MIPS box (except in China), but was released on the Alpha first as OSF/1, OSF AXP and later renamed Tru64 Unix. All of the Alphas (as I recall) ran with the new StorageWorks drives. Oh well, enough nostalgia. >> By the time Linus "invented" Linux that problem had been solved. ... those >> were interesting times. > > "invented" == built upon Andrew Tanenbaum's Minix. Yeah, but who ever heard of Minux except old Unix heads. I doubt many Linux users know what it is/was. ... :) T.T.F.N. William H. Magill # iMac6,1 Core 2 Duo [2.16GHz - 3 GB 667] OS X 10.6.6 # MacBook Pro4.1 Core 2 Duo [2.5GHz - 4GB 667] OS X 10.6.6 # Mac mini Core Duo [1.66 Ghz - 2 GB 667]OS X 10.6.6 # Flat-panel iMac (2.1) [800MHz - Super Drive - 768 Meg] OS X 10.4.11 # PWS433a [Alpha 21164 Rev 7.2 (EV56)- 64 Meg] Tru64 5.1a # XP1000 [Alpha 21264-3 (EV6) - 256 meg] FreeBSD 5.3 # XP1000 [Alpha 21264-A (EV6-7) - 256 meg] FreeBSD 5.3 [email protected] [email protected] [email protected] _______________________________________________ MacOSX-admin mailing list [email protected] http://www.omnigroup.com/mailman/listinfo/macosx-admin
