Re: [9fans] mmap and shared libraries
http://en.wikipedia.org/wiki/Psychological_projection http://en.wikipedia.org/wiki/Lack_of_aesthetical_refinement Bruce Ellis and Noah Evans, please populate the above stub. By the way, Bruce Ellis, do never ever recommend medication to anyone _even in jest_ without providing a visible disclaimer. That can get you caught. --On Sunday, November 09, 2008 12:12 AM +0200 Bruce Ellis [EMAIL PROTECTED] wrote: Noah is dangerously wise. Have you got rid of the smell of mackerel yet? Seriously Eris, you need a good hobby. And if I was your physician I would recommend medication. A SSRI or maybe a simple Benzo. Maybe twice a day, or when required. brucee On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans [EMAIL PROTECTED] wrote: http://en.wikipedia.org/wiki/Psychological_projection On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia [EMAIL PROTECTED] wrote: Little troll, thy baiting f'r fray-- My thoughtless passage has flushed away Am not _I_ a troll like thee, Or art not _thou_ a Goddess like me? Practice your technique, little troll, while you have time to do mischief under the Goddess' nose! --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. Eight billion Windows users can't be wrong. (Can they?)
Re: [9fans] mmap and shared libraries
http://www.apa.org/journals/features/psp7761121.pdf On Sun, Nov 9, 2008 at 8:53 AM, Eris Discordia [EMAIL PROTECTED] wrote: http://en.wikipedia.org/wiki/Psychological_projection http://en.wikipedia.org/wiki/Lack_of_aesthetical_refinement Bruce Ellis and Noah Evans, please populate the above stub. By the way, Bruce Ellis, do never ever recommend medication to anyone _even in jest_ without providing a visible disclaimer. That can get you caught. --On Sunday, November 09, 2008 12:12 AM +0200 Bruce Ellis [EMAIL PROTECTED] wrote: Noah is dangerously wise. Have you got rid of the smell of mackerel yet? Seriously Eris, you need a good hobby. And if I was your physician I would recommend medication. A SSRI or maybe a simple Benzo. Maybe twice a day, or when required. brucee On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans [EMAIL PROTECTED] wrote: http://en.wikipedia.org/wiki/Psychological_projection On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia [EMAIL PROTECTED] wrote: Little troll, thy baiting f'r fray-- My thoughtless passage has flushed away Am not _I_ a troll like thee, Or art not _thou_ a Goddess like me? Practice your technique, little troll, while you have time to do mischief under the Goddess' nose! --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. Eight billion Windows users can't be wrong. (Can they?)
Re: [9fans] mmap and shared libraries
On Sun, Nov 9, 2008 at 5:07 PM, Bruce Ellis [EMAIL PROTECTED] wrote: [...] poor old Mr Peter Enis [...] Wow, this is really sad, but I *just* got that. - Dan C.
Re: [9fans] mmap and shared libraries
I think ericvh has a P.Enis, I know I do. Actually maybe it was Ennis (that one's at home), but it's funny enough. brucee On Mon, Nov 10, 2008 at 12:37 AM, Dan Cross [EMAIL PROTECTED] wrote: On Sun, Nov 9, 2008 at 5:07 PM, Bruce Ellis [EMAIL PROTECTED] wrote: [...] poor old Mr Peter Enis [...] Wow, this is really sad, but I *just* got that. - Dan C.
Re: [9fans] mmap and shared libraries
Sharp, Noah. Shame you and all the 9fans had to get back to real work. I'm trying to avoid it but I have to face reality (again) one day. I'm actually starting to really appreciate Football (Soccer). Though it seems inevitable if you like bars here. I can't but think of poor old Mr Peter Enis when I see mail from Eris. He had an office at the Labs in the nineties (building 6, floor 4 I think). His door plate was very volatile. brucee On Sun, Nov 9, 2008 at 11:16 PM, Noah Evans [EMAIL PROTECTED] wrote: http://www.apa.org/journals/features/psp7761121.pdf On Sun, Nov 9, 2008 at 8:53 AM, Eris Discordia [EMAIL PROTECTED] wrote: http://en.wikipedia.org/wiki/Psychological_projection http://en.wikipedia.org/wiki/Lack_of_aesthetical_refinement Bruce Ellis and Noah Evans, please populate the above stub. By the way, Bruce Ellis, do never ever recommend medication to anyone _even in jest_ without providing a visible disclaimer. That can get you caught. --On Sunday, November 09, 2008 12:12 AM +0200 Bruce Ellis [EMAIL PROTECTED] wrote: Noah is dangerously wise. Have you got rid of the smell of mackerel yet? Seriously Eris, you need a good hobby. And if I was your physician I would recommend medication. A SSRI or maybe a simple Benzo. Maybe twice a day, or when required. brucee On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans [EMAIL PROTECTED] wrote: http://en.wikipedia.org/wiki/Psychological_projection On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia [EMAIL PROTECTED] wrote: Little troll, thy baiting f'r fray-- My thoughtless passage has flushed away Am not _I_ a troll like thee, Or art not _thou_ a Goddess like me? Practice your technique, little troll, while you have time to do mischief under the Goddess' nose! --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. Eight billion Windows users can't be wrong. (Can they?)
Re: [9fans] mmap and shared libraries
Little troll, thy baiting f'r fray-- My thoughtless passage has flushed away Am not _I_ a troll like thee, Or art not _thou_ a Goddess like me? Practice your technique, little troll, while you have time to do mischief under the Goddess' nose! --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. Eight billion Windows users can't be wrong. (Can they?)
Re: [9fans] mmap and shared libraries
http://en.wikipedia.org/wiki/Psychological_projection On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia [EMAIL PROTECTED] wrote: Little troll, thy baiting f'r fray-- My thoughtless passage has flushed away Am not _I_ a troll like thee, Or art not _thou_ a Goddess like me? Practice your technique, little troll, while you have time to do mischief under the Goddess' nose! --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. Eight billion Windows users can't be wrong. (Can they?)
Re: [9fans] mmap and shared libraries
Noah is dangerously wise. Have you got rid of the smell of mackerel yet? Seriously Eris, you need a good hobby. And if I was your physician I would recommend medication. A SSRI or maybe a simple Benzo. Maybe twice a day, or when required. brucee On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans [EMAIL PROTECTED] wrote: http://en.wikipedia.org/wiki/Psychological_projection On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia [EMAIL PROTECTED] wrote: Little troll, thy baiting f'r fray-- My thoughtless passage has flushed away Am not _I_ a troll like thee, Or art not _thou_ a Goddess like me? Practice your technique, little troll, while you have time to do mischief under the Goddess' nose! --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. Eight billion Windows users can't be wrong. (Can they?)
Re: [9fans] mmap and shared libraries
I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. Eight billion Windows users can't be wrong. (Can they?)
Re: [9fans] mmap and shared libraries
Wow what a lively thread. Lots of good information (thanks Ron, Rob et al) and behavioural commentary (thanks Skip) but it seems to be of no avail. As I'm the only one awake in Volos at 10:30am and I have 40 cafes with WiFi to myself I'll waste bandwidth and tell a little story with no stated conclusions. Three 9fans get into a taxi and need to get to Volos Bus Station. They only speak English, Dutch, German, French, Italian, Spanish, Portuguese, and Signing between them - no Greek. How do they get to the bus station for the minimum fare (3 euros)? Sounds like a bad joke, but it really happened. Answer at the end. Fictional scenario: same thing with three Linux/BSD fans who only speak English, Bash and Emacs. Their solution: 1) Whine at the taxi driver - cost 1 euro 2) Shout why doesn't this cab run emacs? - cost 1 euro 3) Download a language package, discover they have incompatible shared libraries - cost 2 euros 4) Shout mmap() me to the bus station - cost 2 euros 5) Take out a phrase book and order a goat and a fishing village - cost 5 euros 6) Get dumped at a distant fishing village where they are savagely beaten and dragged aboard a fishing boat as slaves - cost all their worldly possessions 7) Get dumped back at Volos, because they can't fish without emacs, wearing only shorts that smell of mackerel and one shoe between them - time share the shoe and walk to the bus station - no charge And the answer to the original question? The taxi driver spoke Spanish. The 9fans tip 1 euro. Sorry for wasting your time. brucee -- pages of replies -- elided
Re: [9fans] mmap and shared libraries
grep whining /sys/games/lib/fortunes Why do I have to send this mail every year or so? And who is this Eris dick? brucee On Tue, Nov 4, 2008 at 5:59 PM, ron minnich [EMAIL PROTECTED] wrote: On Tue, Nov 4, 2008 at 7:06 AM, Eris Discordia [EMAIL PROTECTED] wrote: Attempts to live boot Plan 9 on the same machine fail because some 9wacko believes CD-ROM drives must be secondary master or something--and I won't move a jumper to suit a 9wacko's whim; not that I've ever been asked to do that. yep, it's a hackers system. You get to fix it or whine about it, your call. Although I think you've made your preferences clear. Man's got to know his limitations. ron
Re: [9fans] mmap and shared libraries
And what is a 9whacko? I didn't see any at IWP9 but I didn't have a mirror. brucee On Wed, Nov 5, 2008 at 10:32 AM, Lyndon Nerenberg [EMAIL PROTECTED] wrote: And who is this Eris dick? Just a simple first year regexp assignment.
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 10:33 AM, Eris Discordia [EMAIL PROTECTED] wrote: Man's got to know his limitations. Yes, _man_ has got to. That doesn't apply to deities :-P Why do gods that walk the earth invariably act like spoilt brats? Ah, hang on ...
Re: [9fans] mmap and shared libraries
[EMAIL PROTECTED] ~/clms]# ls -l `which vim` -rwxr-xr-x 1 root wheel 1221212 Oct 15 2006 /usr/local/bin/vim C:\Program Files (x86)\Vim\vim71dir gvim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:19 PM 1,585,152 gvim.exe 1 File(s) 1,585,152 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71dir vim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:14 PM 1,372,160 vim.exe 1 File(s) 1,372,160 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71 So what? (and the two latter were _Windows_ binaries). I realize that is utterly unfair. Sort of. Nice of you to realize that. Sort of. --On Tuesday, November 04, 2008 9:21 PM -0800 ron minnich [EMAIL PROTECTED] wrote: On Tue, Nov 4, 2008 at 9:17 PM, Roman Shaposhnik [EMAIL PROTECTED] wrote: A standalone statically linked binary is going to be considerable larger while in flight over data links. ah yes well maybe. [EMAIL PROTECTED]:~/mtelnet ls -l `which emacs` -rwxr-xr-t 1 root root 4587528 2006-06-17 06:59 /usr/bin/emacs [EMAIL PROTECTED]:~/mtelnet ls -l `which acme` -rwxr-xr-x 1 rminnich users 1361257 2008-11-04 12:39 /home/rminnich/plan9/bin/acme [EMAIL PROTECTED]:~/mtelnet I realize that is utterly unfair. Sort of. ron
Re: [9fans] mmap and shared libraries
Why do gods that walk the earth invariably act like spoilt brats? Ah, hang on ... Prolly because a god is only a human's conceited ego. Oh, wait... --On Wednesday, November 05, 2008 10:55 AM + Robert Raschke [EMAIL PROTECTED] wrote: On Wed, Nov 5, 2008 at 10:33 AM, Eris Discordia [EMAIL PROTECTED] wrote: Man's got to know his limitations. Yes, _man_ has got to. That doesn't apply to deities :-P Why do gods that walk the earth invariably act like spoilt brats? Ah, hang on ...
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 4:34 PM, Abhishek Kulkarni [EMAIL PROTECTED] wrote: term% cat pipeto.eris /bin/upas/filter -h $1 $2 'From: Eris Discordia' /dev/null personally, i think eris makes a lot of reasonable points.
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 2:57 AM, Eris Discordia [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] ~/clms]# ls -l `which vim` -rwxr-xr-x 1 root wheel 1221212 Oct 15 2006 /usr/local/bin/vim C:\Program Files (x86)\Vim\vim71dir gvim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:19 PM 1,585,152 gvim.exe 1 File(s) 1,585,152 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71dir vim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:14 PM 1,372,160 vim.exe 1 File(s) 1,372,160 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71 So what? (and the two latter were _Windows_ binaries). I realize that is utterly unfair. Sort of. Nice of you to realize that. Sort of. yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. Ron
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 10:15 AM, ron minnich [EMAIL PROTECTED] wrote: On Wed, Nov 5, 2008 at 2:57 AM, Eris Discordia [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] ~/clms]# ls -l `which vim` -rwxr-xr-x 1 root wheel 1221212 Oct 15 2006 /usr/local/bin/vim C:\Program Files (x86)\Vim\vim71dir gvim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:19 PM 1,585,152 gvim.exe 1 File(s) 1,585,152 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71dir vim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:14 PM 1,372,160 vim.exe 1 File(s) 1,372,160 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71 So what? (and the two latter were _Windows_ binaries). I realize that is utterly unfair. Sort of. Nice of you to realize that. Sort of. yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. Ron How much of that is library support code to make it work? :-)
Re: [9fans] mmap and shared libraries
don't forgot that plan 9 binaries are fully linked while most other systems pull in more code through dynamic linking when the binary is executed. -rob
Re: [9fans] mmap and shared libraries
yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. My intent was, of course, to show your comparison is baseless. It seems you still haven't realized that. You think Plan 9 is great? Sure you know a lot more about it than I do so I think you are entitled to your opinion, but drawing a baseless analogy and ridiculing other OS's--as is common on this mailing list--won't help your cause. I didn't post a listing for a DOS executable. Vim running under cmd.exe (vim.exe) is a normal 32-bit Windows process, only with output to Windows console. The little DOS in every popular Windows ceased to exist like 8 years ago. The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes because the bigger part of the uncompressed 1,585,152 bytes is redundant filler required by the binary format specification. The same happens for the vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE binary format and ELF both require the executable image to contain all initialized but essentially redundant (i.e. zeroed) parts of the data segement. Don't pretend you didn't know that. Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So you have been unfair. No kidding. --On Wednesday, November 05, 2008 10:15 AM -0800 ron minnich [EMAIL PROTECTED] wrote: On Wed, Nov 5, 2008 at 2:57 AM, Eris Discordia [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] ~/clms]# ls -l `which vim` -rwxr-xr-x 1 root wheel 1221212 Oct 15 2006 /usr/local/bin/vim C:\Program Files (x86)\Vim\vim71dir gvim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:19 PM 1,585,152 gvim.exe 1 File(s) 1,585,152 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71dir vim.exe Volume in drive C has no label. Volume Serial Number is B001-B4A3 Directory of C:\Program Files (x86)\Vim\vim71 05/12/2007 12:14 PM 1,372,160 vim.exe 1 File(s) 1,372,160 bytes 0 Dir(s) 5,075,197,952 bytes free C:\Program Files (x86)\Vim\vim71 So what? (and the two latter were _Windows_ binaries). I realize that is utterly unfair. Sort of. Nice of you to realize that. Sort of. yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. Ron
Re: [9fans] mmap and shared libraries
Eris, did you just post the following to slashdot? s/OpenBSD/Plan 9/;s/Theo/9whacko/ and we've got your entire posting history on this mailing list. the similarity is uncanny. Yeah. I'd really like to like OpenBSD. Technically, it's superb. It's smooth, polished, well documented --- it's got a level of consistency that most Linux distros can only hope to dream of. The kernel is well designed and fast, with excellent hardware support. System setup is consistent and well-thought out. Above all, it doesn't confuse easy-to-use with easy-to-learn --- everything is as simple as possible without oversimplifying, which makes it a joy to admin. But then, every time I try to use it, I run up against the OpenBSD developers, who are an arrogant bunch of elitist assholes. In a couple of years, on and off, I think I've seen Theo make a civil reply to someone *once*. Maybe twice. No, I'm not kidding. When you see someone ask what looks to my untutored eye a reasonable question about VMs, and the head developer replies publicly with the words 'You are full of shit' and nothing else (apart from a complete copy of the original message, no snipping), there is something very wrong. Most of the other devs are nearly as bad, and of course there are hordes of groupies who assume that if the people in charge are okay with personal abuse, then it's alright for them, too. Despite this, the actual operating system is definitely worth checking out if you're interested in what a well-designed Unix actually looks like. Linux can learn a lot from it.
Re: [9fans] mmap and shared libraries
less is more. -rob
Re: [9fans] mmap and shared libraries
less is more. If you say so, sir, it must be true. Is it also true that the less I understand of your comment the more meaningful it becomes? --On Wednesday, November 05, 2008 1:12 PM -0800 Rob Pike [EMAIL PROTECTED] wrote: less is more. -rob
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia [EMAIL PROTECTED] wrote: yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. My intent was, of course, to show your comparison is baseless. It seems you still haven't realized that. You think Plan 9 is great? Actually, to keep it simple, I was trying to make a simple point, with a bit of humor intended, and not actually even directed at you,: that code built with shared libraries is not necessarily, or always, smaller than code built otherwise, for programs of similar capability. I don't have an emacs linked statically, however, so I took the nearest materials at hand. It depends on a lot of circumstances. There is the example of xclock which is small with X11 .so, and quite large otherwise. And there, for the case of a clock, shared libraries clearly reduce disk footprint: xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K).. Sadly, the picture changes at run time: clock on plan 9 is 128k in memory, xclock is 4.2M RSS and 10M VSZ. Sic transit gloria .so. Of course, then we hear that well, all that is shared. Hmm. Prove it. FWIW, the whole issue of goodness/badness of shared libraries has never been systematically measured as far as I know -- in terms of performance cost, overhead, throughput, the usual suspects -- or at least I've never seen it done. It would be a lot of work to do it correctly. Might be interesting. The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes So the compressed version is only 50% larger than acme. The point being? because the bigger part of the uncompressed 1,585,152 bytes is redundant filler required by the binary format specification. Does the redundant litter cross the network when I have it mounted via a share and execute the program or not? That was the original discussion. The same happens for the vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE binary format and ELF both require the executable image to contain all initialized but essentially redundant (i.e. zeroed) parts of the data segement. Don't pretend you didn't know that. Wrong for ELF. Simple example from /bin/cat: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align LOAD 0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW 0x1 note that filesize is memory size. I'll let you figure out how it works. This is the problem with your posts. You sound very authoritative, and I'm sure you figure you know all these bits, but you're wrong so often. It's confusing. It would be unfortunate were people to believe you are as right as you think you are. Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So you have been unfair. No kidding. Ah! Now we're into feature comparisons! I'm game. How did you get to 1 in 5 and not 1 in 4.8, or 1 in 6? Of course, you have to take into account that acme's extensions live in /bin or your number is bogus. ron
Re: [9fans] mmap and shared libraries
Is it also true that the less I understand it's as if Choate's twin has come to visit.
Re: [9fans] mmap and shared libraries
less is more. If you say so, sir, it must be true. Is it also true that the less I understand of your comment the more meaningful it becomes? Judging by your posts, the less you know, the more meaningful you consider your opinion, so I'd agree with you here. John
Re: [9fans] mmap and shared libraries
Sadly, the picture changes at run time: clock on plan 9 is 128k in memory, xclock is 4.2M RSS and 10M VSZ. Sic transit gloria .so. Of course, then we hear that well, all that is shared. Hmm. Prove it. I know one thing: shared libraries are employed on every major operating system I have ever heard of. Can all these people be wrong? I don't think so. FWIW, the whole issue of goodness/badness of shared libraries has never been systematically measured as far as I know -- in terms of performance cost, overhead, throughput, the usual suspects -- or at least I've never seen it done. It would be a lot of work to do it correctly. Might be interesting. Title: Dynamic Linking of Software Components Author: Michael Franz YoP: 1997 ABSTRACT The concepts underlying dynamic linking reached maturity through modular operating systems and are being reintroduced in some of the most popular workstation and PC operating systems. [...] http://portal.acm.org/citation.cfm?id=619016.620662coll=GUIDEdl=GUIDECFID=8901601CFTOKEN=68907171 The abstract says it is an overall survey of dynamic linking. I don't have access to the full text and even if I had I wouldn't understand a word of it. You are the CS/CE person. Be the judge, too. So the compressed version is only 50% larger than acme. The point being? That there's a lot of redundancy in the executable principally imposed by the binary format. In case of the FreeBSD vim executable: $ readelf -e /usr/local/bin/vim Reveals 171,365 bytes of redundant data only in .data, .rodata, and .bss sections. $ objdump -s /usr/local/bin/vim Will display tons of zeroes in all other sections except .text. Even RLE could significantly reduce the image size. note that filesize is memory size. I'll let you figure out how it works. I'm not that intelligent but didn't I say it was _initialized_ data that had to be written to executable image? Example from a simple program I wrote to be assembled by Flat Assembler for Win32: section '.data' data readable writeable . . . buffer_ptr dd 00h - pointer reserved, and initialized . . . buffer db 00h dup (?) - buffer reserved, not initialized . . . File size can be less than memory size when you have data reserved but not initialized. That happens in many cases, e.g. when you reserve a buffer. As for ELF: Defining data elements in the bss section is somewhat different from defining them in the data section. Instead of declaring specific data types, you just declare raw segments of memory that are reserved for whatever purpose you need them for. [...] One benefit to declaring data in the bss section is that the data is not included in the executable program. When data is defined in the data section, it must be included in the executable program, since it must be initialized with a specific value. Because the data areas declared in the bss section are not initialized with program data, the memory areas are reserved at runtime, and do not have to be included in the final program. -- Richard Blum, Professional Assembly Language, in the context of using (g)as and ELF format I don't want to sound authoritative but when something is right it is right and there's nothing you or I can do about it. Does the redundant litter cross the network when I have it mounted via a share and execute the program or not? That was the original discussion. I don't know. The size of Emacs executable has as much connection to that question as I do, but it was _you_ who made the bad Emacs joke. Ah! Now we're into feature comparisons! I'm game. How did you get to 1 in 5 and not 1 in 4.8, or 1 in 6? You know very well that the ratio is just an inaccurate measure as the context suggests. 1 to 5 means like your pinkie compared to your entire hand. Trying to context-switch and move from an inaccurate measure to an exact number is just a diversionary technique you have employed for countering an easily demonstrable fact. Namely that Acme is a minimal text editor claiming to be an IDE while Emacs is a behemoth with more features than you could count through a day, and the following night--not that I believe Emacs is any better than Acme. --On Wednesday, November 05, 2008 2:13 PM -0800 ron minnich [EMAIL PROTECTED] wrote: On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia [EMAIL PROTECTED] wrote: yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. My intent was, of course, to show your comparison is baseless. It seems you still haven't realized that. You think Plan 9 is great? Actually, to keep it simple, I was trying to make a simple point, with a bit of humor intended, and not actually even directed at you,: that code built with shared libraries is not
Re: [9fans] mmap and shared libraries
Sadly, the picture changes at run time: clock on plan 9 is 128k in memory, xclock is 4.2M RSS and 10M VSZ. Sic transit gloria .so. Of course, then we hear that well, all that is shared. Hmm. Prove it. I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. FWIW, the whole issue of goodness/badness of shared libraries has never been systematically measured as far as I know -- in terms of performance cost, overhead, throughput, the usual suspects -- or at least I've never seen it done. It would be a lot of work to do it correctly. Might be interesting. Title: Dynamic Linking of Software Components Author: Michael Franz YoP: 1997 ABSTRACT The concepts underlying dynamic linking reached maturity through modular operating systems and are being reintroduced in some of the most popular workstation and PC operating systems. http://portal.acm.org/citation.cfm?id=619016.620662coll=GUIDEdl=GUIDECFID=8901601CFTOKEN=68907171 The abstract says this is an overall survey of dynamic linking. I can't access the full text and I wouldn't understand it even if I could. You are the CS/CE person. Be the judge, too. So the compressed version is only 50% larger than acme. The point being? That there's a lot of redundancy in the executable image principally imposed by the binary format. $ readelf -e /usr/local/bin/vim Reveals over 170 KB of data only in .data, .rodata, and .bss sections. $ objdump -s /usr/local/bin/vim Displays tons of zeroes in all sections except .text. Even RLE could have considerably reduced the image size. Does the redundant litter cross the network when I have it mounted via a share and execute the program or not? That was the original discussion. I don't know. The size of Emacs executable has as much connection to that question as I do, but it was _you_ who made the bad Emacs joke. Wrong for ELF. About ELF: Defining data elements in the bss section is somewhat different from defining them in the data section. Instead of declaring specific data types, you just declare raw segments of memory that are reserved for whatever purpose you need them for. [...] One benefit to declaring data in the bss section is that the data is not included in the executable program. When data is defined in the data section, it must be included in the executable program, since it must be initialized with a specific value. Because the data areas declared in the bss section are not initialized with program data, the memory areas are reserved at runtime, and do not have to be included in the final program. -- Professional Assembly Programming, Richard Blum, in the context of using (g)as and ELF Exactly what I said. I don't want to sound authoritative but when something is right it is right and there's nothing you or I can do about it. So: right for ELF. Executable image size can be less than memory size because there can be _uninitialized_ data reserved in data segments, e.g. when you reserve a buffer but don't care what it initially contains. However, _initialized_ data always and invariably gets written to disk--think of a whole 64K of zeroes. Example from a simple program for Flat Assembler with Win32 target: section '.data' data readable writeable buffer_ptr dd 00h a pointer reserved, and initialized buffer db 00h dup (?) a buffer reserved, but not initialized Ah! Now we're into feature comparisons! I'm game. How did you get to 1 in 5 and not 1 in 4.8, or 1 in 6? You know very well that the ratio is an inaccurate measure as the context suggests. 1 to 5 is like your pinkie to all your fingers. Trying to context-switch and turn a guesstimate into an exact number is only a diversionary technique you have employed to evade an easily demonstrable fact. Namely that Acme is a minimal text editor claiming to be an IDE while Emacs is a behemoth with more features than you could count in a day, and the following night--not that I believe Emacs is any better than Acme. Vim leans more towards Emacs. --On Wednesday, November 05, 2008 2:13 PM -0800 ron minnich [EMAIL PROTECTED] wrote: On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia [EMAIL PROTECTED] wrote: yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. My intent was, of course, to show your comparison is baseless. It seems you still haven't realized that. You think Plan 9 is great? Actually, to keep it simple, I was trying to make a simple point, with a bit of humor intended, and not actually even directed at you,: that code built with shared libraries is not necessarily, or always, smaller than code built otherwise, for programs of similar capability. I don't have an emacs linked statically, however, so I took the nearest materials at hand. It depends on a lot of circumstances. There
Re: [9fans] mmap and shared libraries
Please forgive the repeated messages. It didn't appear in my mail client's Sent view after I hit send. Thought it might have been lost so I re-wrote it.
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 5:53 PM, Eris Discordia [EMAIL PROTECTED] wrote: Please forgive the repeated messages. It didn't appear in my mail client's Sent view after I hit send. Thought it might have been lost so I re-wrote it. i expect you to start littering the web forums and mailing lists dedicated to your mail client software immediately. after all it did cause you discomfort, didn't it? let us know how it goes.
Re: [9fans] mmap and shared libraries
When Sun reported on their first implementation of shared libraries, the paper they presented (I think it was at Usenix) concluded that shared libraries made things bigger and slower, that they were a net loss, and in fact that they didn't save much disk space either. The test case was Xlib, the benefit negative. But the customer expects us to do them so we'll ship it. So yes, every major operating system implements them but that does not mean they are a good idea. Plan 9 was designed to ignore at least some of the received wisdom. -rob
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 4:45 PM, Eris Discordia [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. I know one thing. Every major operating system in the late 1960s knew that card image files were the way to go. Could all those people be wrong? Yes. Sorry, but everyone does it just doesn't hack it. FWIW, the whole issue of goodness/badness of shared libraries has never been systematically measured as far as I know -- in terms of performance cost, overhead, throughput, the usual suspects -- or at least I've never seen it done. It would be a lot of work to do it correctly. Might be interesting. = The abstract says this is an overall survey of dynamic linking. I can't access the full text and I wouldn't understand it even if I could. You are the CS/CE person. Be the judge, too. not what I asked. So the compressed version is only 50% larger than acme. The point being? That there's a lot of redundancy in the executable image principally imposed by the binary format. $ readelf -e /usr/local/bin/vim Reveals over 170 KB of data only in .data, .rodata, and .bss sections. $ objdump -s /usr/local/bin/vim Displays tons of zeroes in all sections except .text. Even RLE could have considerably reduced the image size. ] you're kind of confusing how the file is created with what ELF wants, but that's up to you. Does the redundant litter cross the network when I have it mounted via a share and execute the program or not? That was the original discussion. I don't know. The size of Emacs executable has as much connection to that question as I do, but it was _you_ who made the bad Emacs joke. I.e. You don't know. OK, I'll accept that. Wrong for ELF. About ELF: Defining data elements in the bss section is somewhat different from defining them in the data section. Instead of declaring specific data types, you just declare raw segments of memory that are reserved for whatever purpose you need them for. [...] One benefit to declaring data in the bss section is that the data is not included in the executable program. When data is defined in the data section, it must be included in the executable program, since it must be initialized with a specific value. Because the data areas declared in the bss section are not initialized with program data, the memory areas are reserved at runtime, and do not have to be included in the final program. -- Professional Assembly Programming, Richard Blum, in the context of using (g)as and ELF Exactly what I said. I don't want to sound authoritative but when something is right it is right and there's nothing you or I can do about it. So: right for ELF. I see your confusion, and it is common, and I have made this mistake too. ELF is like that :-) And readelf can make it worse. You need to understand the difference between sections and segments. From the point of view of the program loader, there is no such thing as a ..bss. From the point of view of the linker, there are .bss segments. But those don't have lots of meaning for a program loader. In fact program loaders that look at sections are broken. So you're going to need to go back to the standard, but most people are: I once worked with code that was created by a big company that mistakenly parsed the sections, not the segments, of an executable. It was a mess. But think of it this way: You can have a valid ELF executable that has no sections. Sections have nothing to do with running the program. So, Wrong for ELF. Sorry. But it's an incredibly common mistake. Example from a simple program for Flat Assembler with Win32 target: section '.data' data readable writeable buffer_ptr dd 00h a pointer reserved, and initialized buffer db 00h dup (?) a buffer reserved, but not initialized Trust me. I don't care about win32 targets. Ah! Now we're into feature comparisons! I'm game. How did you get to 1 in 5 and not 1 in 4.8, or 1 in 6? You know very well that the ratio is an inaccurate measure as the context suggests. 1 to 5 is like your pinkie to all your fingers. Trying to context-switch and turn a guesstimate into an exact number is only a diversionary technique you have employed to evade an easily demonstrable fact. Namely that Acme is a minimal text editor claiming to be an IDE while Emacs is a behemoth with more features than you could count in a day, and the following night--not that I believe Emacs is any better than Acme. Vim leans more towards Emacs. So I guess that acme has more features than vi, since it's extensions are any executable; you guess not. My claim is you don't actually understand acme. Let's leave it at that. Except, uh, who claimed acme was an IDE? Just wondering. It wasn't me. ron
Re: [9fans] mmap and shared libraries
I know one thing. Every major operating system in the late 1960s knew that card image files were the way to go. Could all those people be wrong? Yes. Sorry, but everyone does it just doesn't hack it. it's the chewbacca proof. - erik
Re: [9fans] mmap and shared libraries
File size can be less than memory size when you have data reserved but not initialized. That happens in many cases, e.g. when you reserve a buffer. One benefit to declaring data in the bss section is that the data is not included in the executable program. When data is defined in the data section, it must be included in the executable program, since it must be initialized with a specific value. Because the data areas declared in the bss section are not initialized with program data, the memory areas are reserved at runtime, and do not have to be included in the final program. i don't think uninitialized means what you think it does. for the benefit of those who might be confused by this largely purposeful misdirection, the bss is initialized to zero. the c standard specifies that external declarations without explicit initialization are initialized to zero. this is intialized data that doesn't appear in the executable. because i'm pedantic, i'll point out that zero has always been a specific value. - erik
Re: [9fans] mmap and shared libraries
On Wed, Nov 5, 2008 at 6:48 PM, Eris Discordia [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. http://en.wikipedia.org/wiki/Argumentum_ad_populum
Re: [9fans] mmap and shared libraries
On Nov 5, 2008, at 2:13 PM, ron minnich wrote: On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia [EMAIL PROTECTED] wrote: yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is about 1/2 M. Vim on DOS is 3x larger? impressive. My intent was, of course, to show your comparison is baseless. It seems you still haven't realized that. You think Plan 9 is great? Sadly, the picture changes at run time: clock on plan 9 is 128k in memory, xclock is 4.2M RSS and 10M VSZ. Sic transit gloria .so. Of course, then we hear that well, all that is shared. Hmm. Prove it. Exactly! On Linux with ld.so accepting relocations (think non-PIC code) in .text the appearance of some .so's being shared is highly deceiving. Solaris is better in that regard, but the pressure from userland community expecting their project to just work is mounting. Thanks, Roman.
Re: [9fans] mmap and shared libraries
Eris Discordia wrote: I know one thing I doubt that.
Re: [9fans] mmap and shared libraries
Thanks for the information. --On Wednesday, November 05, 2008 5:23 PM -0800 Rob Pike [EMAIL PROTECTED] wrote: When Sun reported on their first implementation of shared libraries, the paper they presented (I think it was at Usenix) concluded that shared libraries made things bigger and slower, that they were a net loss, and in fact that they didn't save much disk space either. The test case was Xlib, the benefit negative. But the customer expects us to do them so we'll ship it. So yes, every major operating system implements them but that does not mean they are a good idea. Plan 9 was designed to ignore at least some of the received wisdom. -rob
Re: [9fans] mmap and shared libraries
Sorry, but everyone does it just doesn't hack it. also, everyone does it is an excuse that no one over the age 7 should use. imitating blindly -- a.k.a monkey-see-monkey-do (apologies to monkeys) -- seems to happen when we are unaware, undisciplined, lazy or panicked. it is how stampedes happen. it's ok if you're in a herd, can't see much and trying to avoid a predator; not so for designing software. a good example can be found in an episode of a UW TV program called behind the code, produced by MS and UW, where one of the original developers of NT (hardware and drivers) recalls a kernel crash that was caused by bad MS sample code that ended up in several driver by different vendors.
Re: [9fans] mmap and shared libraries
Sorry, but everyone does it just doesn't hack it. Common-sensically it does, but if you say it doesn't I concede. not what I asked. It seems to answer your question by implying there's no question regarding the use or disuse of shared libraries only regarding the strategy (and that back in 1997). But I can't read beyond the abstract so I can't tell if it really goes that far. Incidentally, it references some Wirth papers. you're kind of confusing how the file is created with what ELF wants, but that's up to you. Perhaps, I'll try to read up on it. You need to understand the difference between sections and segments. From the point of view of the program loader, there is no such thing as a ..bss. I believe I understand the difference. The loader may put .bss, .data, .rodata, and some other data sections around the same place in memory and decide about the order and relative placement of each. That doesn't change anything about the binary format, however. Only when the image is being loaded .bss grows to its true size. On the other hand, .data is more or less the same in memory or on disk. But think of it this way: You can have a valid ELF executable that has no sections. Sections have nothing to do with running the program. Initialized data has to be placed somewhere in the image, anyway. So, Wrong for ELF. Sorry. But it's an incredibly common mistake. I'll try to learn more about it (and probably report back... okay, I won't). So I guess that acme has more features than vi, since it's extensions are any executable; you guess not. My claim is you don't actually understand acme. That's a probable cause but only probable. Let's leave it at that. Except, uh, who claimed acme was an IDE? Just wondering. It wasn't me. Well, leave it at that. By the way, the programmer's do-all is an IDE I presume. Finally, thanks for bearing with me and supplying things I didn't know. --On Wednesday, November 05, 2008 5:25 PM -0800 ron minnich [EMAIL PROTECTED] wrote: On Wed, Nov 5, 2008 at 4:45 PM, Eris Discordia [EMAIL PROTECTED] wrote: I know one thing: every major operating system I have ever heard of leverages shared libraries. Can all those people be wrong? I don't think so. I know one thing. Every major operating system in the late 1960s knew that card image files were the way to go. Could all those people be wrong? Yes. Sorry, but everyone does it just doesn't hack it. FWIW, the whole issue of goodness/badness of shared libraries has never been systematically measured as far as I know -- in terms of performance cost, overhead, throughput, the usual suspects -- or at least I've never seen it done. It would be a lot of work to do it correctly. Might be interesting. = The abstract says this is an overall survey of dynamic linking. I can't access the full text and I wouldn't understand it even if I could. You are the CS/CE person. Be the judge, too. not what I asked. So the compressed version is only 50% larger than acme. The point being? That there's a lot of redundancy in the executable image principally imposed by the binary format. $ readelf -e /usr/local/bin/vim Reveals over 170 KB of data only in .data, .rodata, and .bss sections. $ objdump -s /usr/local/bin/vim Displays tons of zeroes in all sections except .text. Even RLE could have considerably reduced the image size. ] you're kind of confusing how the file is created with what ELF wants, but that's up to you. Does the redundant litter cross the network when I have it mounted via a share and execute the program or not? That was the original discussion. I don't know. The size of Emacs executable has as much connection to that question as I do, but it was _you_ who made the bad Emacs joke. I.e. You don't know. OK, I'll accept that. Wrong for ELF. About ELF: Defining data elements in the bss section is somewhat different from defining them in the data section. Instead of declaring specific data types, you just declare raw segments of memory that are reserved for whatever purpose you need them for. [...] One benefit to declaring data in the bss section is that the data is not included in the executable program. When data is defined in the data section, it must be included in the executable program, since it must be initialized with a specific value. Because the data areas declared in the bss section are not initialized with program data, the memory areas are reserved at runtime, and do not have to be included in the final program. -- Professional Assembly Programming, Richard Blum, in the context of using (g)as and ELF Exactly what I said. I don't want to sound authoritative but when something is right it is right and there's nothing you or I can do about it. So: right for ELF. I see your confusion, and it is common, and I have made this mistake too. ELF is like that :-) And readelf can make it worse. You need to understand the difference between sections and segments. From
Re: [9fans] mmap
time travel plots reminds me of an obscure but splendid czech film that i've only seen once http://filmjournal.net/czech/2006/09/18/tomorrow-ill-wake-up-and-scald-myself-with-tea/ but have yet to find on DVD. it is very funny.
Re: [9fans] mmap and shared libraries
I can run Plan 9 quite nicely in 128 MB of RAM. In the same amount of memory FreeBSD is paging nightmare, despite it's wonderfully complex shared library environment. You're wrong. Case in point: my FreeBSD 6.2-RELEASE installation on a 233 MHz PII (one of those Slot 1 processors) with 128 MB of RAM runs just fine and serves well the purpose of 24x7 downloading and serving FTP; home scale, of course. Attempts to live boot Plan 9 on the same machine fail because some 9wacko believes CD-ROM drives must be secondary master or something--and I won't move a jumper to suit a 9wacko's whim; not that I've ever been asked to do that. --On Monday, November 03, 2008 5:23 PM -0800 Lyndon Nerenberg [EMAIL PROTECTED] wrote: A thought ... Shared libraries do 2 possibly useful things: 1) save space 2) stop you having to re-link when a new library is released. Now 2) doesn't really happen anyway, due to .so versioning hell, so we're left with 1) ... I can run Plan 9 quite nicely in 128 MB of RAM. In the same amount of memory FreeBSD is paging nightmare, despite it's wonderfully complex shared library environment. Oh, and Solaris won't even boot with less 512 MB these days. There is no space shortage on Plan 9 that's looking for a solution.
Re: [9fans] mmap
On Mon, Nov 3, 2008 at 11:15 PM, Andrew Simmons [EMAIL PROTECTED] wrote: I don't want to imply that Ron is quite such an old fart as me, but somehow I don't get the impression that he was a kid in 1981, when Time Bandits came out. Ron, if you could give some clue as to when you saw the movie, I'm pretty sure that the group could mount a co-ordinated effort to identify it. It had to be 1963. I am doubtless older than you, my common experience nowadays is being older than everyone. But so much more charming! ron
Re: [9fans] mmap and shared libraries
On Tue, Nov 4, 2008 at 7:06 AM, Eris Discordia [EMAIL PROTECTED] wrote: Attempts to live boot Plan 9 on the same machine fail because some 9wacko believes CD-ROM drives must be secondary master or something--and I won't move a jumper to suit a 9wacko's whim; not that I've ever been asked to do that. yep, it's a hackers system. You get to fix it or whine about it, your call. Although I think you've made your preferences clear. Man's got to know his limitations. ron
Re: [9fans] mmap
On Nov 4, 2008, at 9:02 AM, ron minnich wrote: On Mon, Nov 3, 2008 at 11:15 PM, Andrew Simmons [EMAIL PROTECTED] wrote: I don't want to imply that Ron is quite such an old fart as me, but somehow I don't get the impression that he was a kid in 1981, when Time Bandits came out. Ron, if you could give some clue as to when you saw the movie, I'm pretty sure that the group could mount a co-ordinated effort to identify it. It had to be 1963. I am doubtless older than you, my common experience nowadays is being older than everyone. But so much more charming! I think the movie is The Time Travelers. I remember seeing it in the early 60's. Maybe I'm getting too old... Hey you kids! Get off of my lawn. Kim
Re: [9fans] mmap
On Tue, 04 Nov 2008 20:15:04 +1300 Andrew Simmons [EMAIL PROTECTED] wrote: I don't want to imply that Ron is quite such an old fart as me, but somehow I don't get the impression that he was a kid in 1981, when Time Bandits came out. Ron, if you could give some clue as to when you saw the movie, I'm pretty sure that the group could mount a co-ordinated effort to identify it. Sounds like The Time Travelers
Re: [9fans] mmap
On Mon, Nov 3, 2008 at 8:43 PM, Enrico Weigelt [EMAIL PROTECTED] wrote: * ron minnich [EMAIL PROTECTED] wrote: I wish I could remember. It had the usual guys in silvery suits. They walk through a frame and are back in time. Key point was, at the end, that they ended up escaping but for reasons unknown, walking back through the frame -- bad idea. Time bandits ? Don't touch it... it's pure evil. cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ cellphone: +49 174 7066481 email: [EMAIL PROTECTED] skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [9fans] mmap and shared libraries
On Nov 3, 2008, at 5:16 AM, [EMAIL PROTECTED] wrote: A thought ... Shared libraries do 2 possibly useful things: 1) save space 2) stop you having to re-link when a new library is released. Now 2) doesn't really happen anyway, due to .so versioning hell, so we're left with 1) ... I know it's kind-of hacky and unstructured (how do you know the venti block size?), but could you not block-align the library members in the .a, so venti would do the sharing on disk for you? As a further step, you could also page align the text segments of the library members, then implement a venti-like scheme for read-only pages in memory, so that when the executable loads, it gets any common library pages shared. I was thinking about exactly the same approach when I was contemplating the evils of dynamic linking. It seems that if you link statically and you use FS to help you coalesce identical blocks (on storage devices and in RAM) you get all of the potential benefits of a dynamically linked binary except one -- distribution. A standalone statically linked binary is going to be considerable larger while in flight over data links. P.S. Sorry not to make Volos: where are the rest of the photos? Yeah -- where are they?!?! ;-) Thanks, Roman.
Re: [9fans] mmap and shared libraries
A standalone statically linked binary is going to be considerable larger while in flight over data links. But that static binary only flies once, geting sucked into memory with a (mostly) simple bcopy equiv at process launch time. Shared memory regimes thrash the living daylights out of MMUs and PTLBs, on each and every context switch.
Re: [9fans] mmap and shared libraries
A thought ... Shared libraries do 2 possibly useful things: 1) save space 2) stop you having to re-link when a new library is released. Now 2) doesn't really happen anyway, due to .so versioning hell, so we're left with 1) ... I know it's kind-of hacky and unstructured (how do you know the venti block size?), but could you not block-align the library members in the .a, so venti would do the sharing on disk for you? As a further step, you could also page align the text segments of the library members, then implement a venti-like scheme for read-only pages in memory, so that when the executable loads, it gets any common library pages shared. Dave. P.S. Sorry not to make Volos: where are the rest of the photos?
Re: [9fans] mmap and shared libraries
[EMAIL PROTECTED] wrote: A thought ... Shared libraries do 2 possibly useful things: 1) save space 2) stop you having to re-link when a new library is released. Now 2) doesn't really happen anyway, due to .so versioning hell, so we're left with 1) ... I know it's kind-of hacky and unstructured (how do you know the venti block size?), but could you not block-align the library members in the .a, so venti would do the sharing on disk for you? Look at this: http://gsoc.cat-v.org/people/mjl/blog//2007-08-06-1_Rabin_fingerprints As a further step, you could also page align the text segments of the library members, then implement a venti-like scheme for read-only pages in memory, so that when the executable loads, it gets any common library pages shared. Dave. P.S. Sorry not to make Volos: where are the rest of the photos?
Re: [9fans] mmap and shared libraries
A thought ... Shared libraries do 2 possibly useful things: 1) save space 2) stop you having to re-link when a new library is released. Now 2) doesn't really happen anyway, due to .so versioning hell, so we're left with 1) ... I can run Plan 9 quite nicely in 128 MB of RAM. In the same amount of memory FreeBSD is paging nightmare, despite it's wonderfully complex shared library environment. Oh, and Solaris won't even boot with less 512 MB these days. There is no space shortage on Plan 9 that's looking for a solution.
Re: [9fans] mmap and shared libraries
On Mon, Nov 3, 2008 at 07:16, [EMAIL PROTECTED] wrote: A thought ... Shared libraries do 2 possibly useful things: 1) save space 2) stop you having to re-link when a new library is released. i can see how relinks are painful with gnu-style build systems where you need to run ./configure and recurse thru endless Makefiles. but under the much cleaner plan 9 source tree, such things take seconds. plan 9 doesn't really have that many libraries either. my (probably naïve) impression is that much of what can be done with dynamically linked libraries can be done with pipes or layered filesystems also linking at loadtime or runtime slows things down. you might not notice interactively, but in shell scripts you certainly would. that is one of the reasons why the unix world uses shell scripts to a lesser extent. there are other advantages to static linking, but they slip my mind for now losing the advantages of static linking in order to duplicate functionality that can be gotten from pipes and filesystems doesn't seem like such a good tradeoff to me -- m
Re: [9fans] mmap
* ron minnich [EMAIL PROTECTED] wrote: I wish I could remember. It had the usual guys in silvery suits. They walk through a frame and are back in time. Key point was, at the end, that they ended up escaping but for reasons unknown, walking back through the frame -- bad idea. Time bandits ? cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ cellphone: +49 174 7066481 email: [EMAIL PROTECTED] skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [9fans] mmap
I don't want to imply that Ron is quite such an old fart as me, but somehow I don't get the impression that he was a kid in 1981, when Time Bandits came out. Ron, if you could give some clue as to when you saw the movie, I'm pretty sure that the group could mount a co-ordinated effort to identify it. 2008/11/4 Enrico Weigelt [EMAIL PROTECTED]: * ron minnich [EMAIL PROTECTED] wrote: I wish I could remember. It had the usual guys in silvery suits. They walk through a frame and are back in time. Key point was, at the end, that they ended up escaping but for reasons unknown, walking back through the frame -- bad idea. Time bandits ? cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ cellphone: +49 174 7066481 email: [EMAIL PROTECTED] skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [9fans] mmap
* Roman V. Shaposhnik [EMAIL PROTECTED] wrote: On Wed, 2008-07-30 at 17:29 +0200, Enrico Weigelt wrote: Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. What aspect of shared libraries are you aching for? Dynamic linking or the dynamic loading? 3rd: Sharing pages. Well, this perhaps also could be done if the kernel would be able to detect equal pages and automatically map them together (maybe w/ copy-on-write again). BTW mmap() is also nice for creating shared memory between (local) processes. For example RDBMS'es can get a huge benetit from this. cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ cellphone: +49 174 7066481 email: [EMAIL PROTECTED] skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [9fans] mmap
* Roman V. Shaposhnik [EMAIL PROTECTED] wrote: On Wed, 2008-07-30 at 17:29 +0200, Enrico Weigelt wrote: Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. What aspect of shared libraries are you aching for? Dynamic linking or the dynamic loading? 3rd: Sharing pages. segment(3) already provides this. - erik
Re: [9fans] mmap
On Sun, Nov 2, 2008 at 5:50 PM, erik quanstrom [EMAIL PROTECTED] wrote: * Roman V. Shaposhnik [EMAIL PROTECTED] wrote: On Wed, 2008-07-30 at 17:29 +0200, Enrico Weigelt wrote: Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. What aspect of shared libraries are you aching for? Dynamic linking or the dynamic loading? 3rd: Sharing pages. segment(3) already provides this. I still remember this science fiction movie from when I was a kid. Time travelers. At the end of the movie, you realized that they were ending at the point where they were started, stuck in a loop, oh no! and here we are at mmap again. ron
Re: [9fans] mmap
* erik quanstrom [EMAIL PROTECTED] wrote: * Roman V. Shaposhnik [EMAIL PROTECTED] wrote: On Wed, 2008-07-30 at 17:29 +0200, Enrico Weigelt wrote: Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. What aspect of shared libraries are you aching for? Dynamic linking or the dynamic loading? 3rd: Sharing pages. segment(3) already provides this. hmm, so segment(3)+segattach(2) can be seen as a kind of a frontend for mmap() ;-) But now I'm curious how executables and shared libraries are actually handled on plan9. cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ cellphone: +49 174 7066481 email: [EMAIL PROTECTED] skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [9fans] mmap
On Sun, Nov 2, 2008 at 8:18 PM, Enrico Weigelt [EMAIL PROTECTED] wrote: But now I'm curious how executables and shared libraries are actually handled on plan9. what's a shared library? Executables: /sys/src/9/ Check it out, it's short and sweet. ron
Re: [9fans] mmap
I still remember this science fiction movie from when I was a kid. which one was that? it sounds more interesting than mmap.
Re: [9fans] mmap
On Sun, Nov 2, 2008 at 11:29 PM, Charles Forsyth [EMAIL PROTECTED] wrote: I still remember this science fiction movie from when I was a kid. which one was that? it sounds more interesting than mmap. I wish I could remember. It had the usual guys in silvery suits. They walk through a frame and are back in time. Key point was, at the end, that they ended up escaping but for reasons unknown, walking back through the frame -- bad idea. ron
Re: [9fans] mmap
On Sat, Aug 2, 2008 at 6:22 AM, Richard Miller [EMAIL PROTECTED] wrote: has hardware channels. And you can call from channel and execute code being sent down a channel to you from another cpu. ... it's a very interesting architecture, to say the least. For me anyway the most novel thing I've seen in a while. Interesting but not novel. Remember the transputer, circa 1985? Hardware channels, both inter- and intra-chip. The normal way of loading code was to read it down a link from another processor - used for bootstrapping an array of transputers, but also could be used for dynamic propagation of processes through the network. yes, but IIRC the transputer did not have the ability to execute code from a channel in the way the ambric does -- literally having a PC that includes a channel ID. There are a few other wrinkles to the ambric that make it a little neater than the transputer. Ambric is pretty well aware of what the transputer did. rno
Re: [9fans] mmap
Ambric is pretty well aware of what the transputer did. Glad to hear it - there were some good ideas behind the transputer which are worth recycling. From your description it sounds like the ambric has some interesting new refinements as well.
Re: [9fans] mmap
On Wed, Jul 30, 2008 at 18:36, Roman V. Shaposhnik [EMAIL PROTECTED] wrote: As Russ, quite rightfully, pointed out: mmap() means different things to different people. The tragic part is, that it tries to do lots of things but it doesn't do anything particularly well. Personally, my experience of trying to use mmap() as a useful abstraction for the CPU's MMU was the last straw. It can't do even that reliably and in a portable fashion. Not to digress, but I was even more surprised to learn that there's not a single API on UNIX that can: http://thread.gmane.org/gmane.os.netbsd.devel.kernel/6392/focus=6457 So, what mmap() (the way it is done on UNIX) is good for? Here's my personal list. Feel free to add (and suggest alternatives on systems lacking mmap() such as Plan9): * a *lazy* way of handling highly irregular I/O over large files. Cases, where you can't really tell which parts of the file are going to be needed. The best example here is mmap() on exec. You don't have to read() all of .text if the actual execution path only takes you to a couple of routines. * an optimization for regular I/O. To some extent, I've always wondered why read always takes its second argument -- a lot of times I don't really care where the buffer with the data I need ends up in my address space. That's pretty much it. Everything else, feels like a hack in a dire need of a better abstraction. SBCL uses it to kick out system vm out of managing it's memory. At least the version I got on linux calls mmap() with MAP_ANONYMOUS, rwx permissions and I think MAP_FIXED. Unfortunately, when you disable overcommit it fails to start on my machine, because I clearly don't have 8GB of free memoryswap for it (I total at 2.5 GB w/ swap). I wish someone gave a plain interface to just took control of memory space... I also had seen a simple malloc() implementation (targeted at people who want to write their own systems) that had a simple form of mmap/unmap calls (without mapping of files) as it's only requirement. If it's the only kernel-exported API for managing memory, it can make sense. I think that was done in L4 API, which organizes it's processes by address spaces (the distinction between what we call a process and thread being only their address spaces - a process was a set of threads grouped by address space). A process could also implement it's own pager and create processes that used it's address space (basically what 9vx does, as you could also install appropriate syscall trampolines). As a counter-example, MS Windows' Notepad.exe is one of the worst offenders when it comes to mmap(). To put it simply, the reason it doesn't work on large files is that it mmaps them. ALWAYS. It's even stated in IFS SDK, that implementation of memory mapping mechanism is required to get notepad to work on the filesystem in question. Thanks, Roman. -- Paweł Lasek
Re: [9fans] mmap
here is a thought: the kernel does mmap for code/data. This is because we think of a file as a segment of data that somehow maps well to a segment of memory. You wouldn't execute code from a stream, now, would you? Well, this: http://www.ambric.com/ has hardware channels. And you can call from channel and execute code being sent down a channel to you from another cpu. There's no real analogue to this in any OS I've used for a while ... You could write the mcilroy sieve in this very directly. You could even, when starting a new thread, push the code down to the next cpu. And that cpu is paused in a call from channel until it gets its code. A nice way to keep them idle until you need them. it's a very interesting architecture, to say the least. For me anyway the most novel thing I've seen in a while. ron
Re: [9fans] mmap
On Thu, Jul 31, 2008 at 5:32 PM, ron minnich [EMAIL PROTECTED] wrote: here is a thought: the kernel does mmap for code/data. This is because we think of a file as a segment of data that somehow maps well to a segment of memory. You wouldn't execute code from a stream, now, would you? Well, this: http://www.ambric.com/ has hardware channels. And you can call from channel and execute code being sent down a channel to you from another cpu. There's no real analogue to this in any OS I've used for a while ... You could write the mcilroy sieve in this very directly. You could even, when starting a new thread, push the code down to the next cpu. And that cpu is paused in a call from channel until it gets its code. A nice way to keep them idle until you need them. it's a very interesting architecture, to say the least. For me anyway the most novel thing I've seen in a while. The ARM 7 and ARM 9 procs in the Nintendo DS can talk back and forth via a FIFO at a pretty low level :-) http://www.double.co.nz/nintendo_ds/nds_develop7.html Dave ron
Re: [9fans] mmap
* Venkatesh Srinivas [EMAIL PROTECTED] wrote: ... redirecting back to 9fans ;-P As far as interfaces go, mmap() is pretty tragic - the underlying translation structures can express more interesting things, some of which are even worth doing. Well, the biggest problem, IMHO are the incompatible semantics between platforms, and some things IMHO aren't needed at all. What I expect from an clean mmap() is: * map in a given region of some file into process's address space * mapping types: a) readonly b) readwrite c) private / copy-on-write * either let the app or the kernel decide on start address * optional hints on intended access patterns (eg. if it might be wise to do read-ahead) Ah, and there should be a special (size-limited) kernel device which holds evrything in RAM exclusively (never get swapped out) for holding critical security information. There have even been OSes that let userland apps play with their address spaces in far more interesting ways - KeyKOS and EROS come to mind. And they were even fast, or something. hmm, never heared about them. what are they doing at this point ? In a system like Plan 9, where your file servers are on the other side of a 9P link, this mmap thing seems dubious. If what you want is the convenience that you get from having all the bytes in memory, reading them all in wouldn't be too hard. mmap()s magic really arises when you have a page-cache-like-thing. Well, mmap() is a tricky thing since it breaks common filesystem semantics. So it can only be done, and only makes sense on specific kind of files/devices which are only acessed block-wise. So the absolute MUST is that the underlying file supports (at least block- wise) random access - mmap()'ed streams are completely nonsense. Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ cellphone: +49 174 7066481 email: [EMAIL PROTECTED] skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [9fans] mmap
On Wed, 2008-07-30 at 17:29 +0200, Enrico Weigelt wrote: Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. What aspect of shared libraries are you aching for? Dynamic linking or the dynamic loading? Thanks, Roman.
Re: [9fans] mmap
On Wed, Jul 30, 2008 at 11:29 AM, Enrico Weigelt [EMAIL PROTECTED] wrote: Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. Actually, almost any kernel support for shared libraries will need something like mmap() internally. I forget who said it, and the local firewall won't allow me to search the online copy of /sys/games/lib/fortunes, but there should be a line there about Linux having 200+ system calls, most of them emulatable with mmap(). --Joel
Re: [9fans] mmap
On Tue, 2008-07-29 at 12:28 -0400, Venkatesh Srinivas wrote: As far as interfaces go, mmap() is pretty tragic - the underlying translation structures can express more interesting things, some of which are even worth doing. I can't agree more. The way I look at it is that mmap() seems to be the answer but nobody ever bothered to ask the question it is supposed to answer. In a system like Plan 9, where your file servers are on the other side of a 9P link, this mmap thing seems dubious. If what you want is the convenience that you get from having all the bytes in memory, reading them all in wouldn't be too hard. mmap()s magic really arises when you have a page-cache-like-thing. As Russ, quite rightfully, pointed out: mmap() means different things to different people. The tragic part is, that it tries to do lots of things but it doesn't do anything particularly well. Personally, my experience of trying to use mmap() as a useful abstraction for the CPU's MMU was the last straw. It can't do even that reliably and in a portable fashion. Not to digress, but I was even more surprised to learn that there's not a single API on UNIX that can: http://thread.gmane.org/gmane.os.netbsd.devel.kernel/6392/focus=6457 So, what mmap() (the way it is done on UNIX) is good for? Here's my personal list. Feel free to add (and suggest alternatives on systems lacking mmap() such as Plan9): * a *lazy* way of handling highly irregular I/O over large files. Cases, where you can't really tell which parts of the file are going to be needed. The best example here is mmap() on exec. You don't have to read() all of .text if the actual execution path only takes you to a couple of routines. * an optimization for regular I/O. To some extent, I've always wondered why read always takes its second argument -- a lot of times I don't really care where the buffer with the data I need ends up in my address space. That's pretty much it. Everything else, feels like a hack in a dire need of a better abstraction. Thanks, Roman.
Re: [9fans] mmap
Roman V. Shaposhnik wrote: Personally, my experience of trying to use mmap() as a useful abstraction for the CPU's MMU was the last straw. It can't do even that reliably and in a portable fashion. Not to digress, but I was even more surprised to learn that there's not a single API on UNIX that can: http://thread.gmane.org/gmane.os.netbsd.devel.kernel/6392/focus=6457 Amen. I've had to get our OS team to shoehorn in some remap() calls that let me point two(many) virtual addresses to a common physical address. It's disgusting that the process for doing this with mmap requires /proc/#/mem and other scary things. Does anyone have pointers to *nice* interfaces for doing this? I don't care which OS, but I'd like to not have the implementation I'm requesting suck because I missed some gotchas that someone else already hit. Thanks, Paul
Re: [9fans] mmap
Joel C. Salomon wrote: On Wed, Jul 30, 2008 at 11:29 AM, Enrico Weigelt [EMAIL PROTECTED] wrote: Convenience is one point (sometimes be a big point), but another important one is sharing. Without mmap(), an (real) shared library support most likely will require special kernel support. Actually, almost any kernel support for shared libraries will need something like mmap() internally. I forget who said it, and the local firewall won't allow me to search the online copy of /sys/games/lib/fortunes, but there should be a line there about Linux having 200+ system calls, most of them emulatable with mmap(). what nonsense is that? --Joel cinap
Re: [9fans] mmap
On Wed, Jul 30, 2008 at 12:25 PM, Joel C. Salomon [EMAIL PROTECTED] wrote: I forget who said it, Found it in http://9fans.net/archive/2002/08/130: On Tue, 13 Aug 2002 07:43:45 -0400, David Gordon Hogan [EMAIL PROTECTED] wrote: On freebsd and Linux, exec happens via an mmap (more or less). snip So [gigantic leap here], not only does Linux have ~250 system calls, but most of them can be emulated with mmap? --Joel
Re: [9fans] mmap
* Russ Cox [EMAIL PROTECTED] wrote: Hi, big_snip / I don't know much of native plan9 (only using plan9port), but IMHO an full mmap() is a really nice thing. It can make a lot things easier if you just map the whole file into the process' memory and let the kernel handle the actual IO. Some time ago I hacked up a little XML-based database (just to show myself that it needn't to be that ugly slow as Xindice ;-)), and being able to conveniently work with pointers instead of ugly seek() made really fun ;-P cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ cellphone: +49 174 7066481 email: [EMAIL PROTECTED] skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [9fans] mmap
It can make a lot things easier if you just map the whole file into the process' memory and let the kernel handle the actual IO. the word superficially should be in there somewhere.
Re: [9fans] mmap
On Tue, 29 Jul 2008 12:52:14 +0400, Enrico Weigelt [EMAIL PROTECTED] wrote: Hi! an full mmap() is a really nice thing. It can make a lot things easier if you just map the whole file into the process' memory and let the kernel handle the actual IO. Yes, it is comfortable. But just think a bit - what will you do in the mmap implementation when you had mapped a remote file (in Plan9 you can't be sure some file is local or it is really just a file), and the connection has just been broken? Surprise! For example, in Windows you will have an access violation. It is very funny to rewrite a code for using the good old file i/o instead of mmap. In Linux you can't map a non-regular file. Yes, you can read entire the file into the memory, but a size of the file can be very big and you will destroy the main advantage of mmap - the reading of only needed pieces of the file, not entire. I beleive, mmap is not for distributed systems. -- Best regards, santucco
Re: [9fans] mmap
Yes, it is comfortable. where's jim when you need him? But just think a bit - what will you do in the mmap implementation when you had mapped a remote file (in Plan9 you can't be sure some file is local or it is really just a file), and the connection has just been broken? Surprise! you can't make the assumption that a file is local in *ix, either. - erik
Re: [9fans] mmap
On Tue, Jul 29, 2008 at 8:19 AM, erik quanstrom [EMAIL PROTECTED] wrote: you can't make the assumption that a file is local in *ix, either. in fact, for the last 20 years, every program run on a sunos/solaris machine has used mmap for the exec. ron
Re: [9fans] mmap
On Tue, Jul 29, 2008 at 8:04 AM, Alexander Sychev [EMAIL PROTECTED]wrote: On Tue, 29 Jul 2008 12:52:14 +0400, Enrico Weigelt [EMAIL PROTECTED] wrote: Hi! an full mmap() is a really nice thing. It can make a lot things easier if you just map the whole file into the process' memory and let the kernel handle the actual IO. Yes, it is comfortable. But just think a bit - what will you do in the mmap implementation when you had mapped a remote file (in Plan9 you can't be sure some file is local or it is really just a file), and the connection has just been broken? Surprise! For example, in Windows you will have an access violation. It is very funny to rewrite a code for using the good old file i/o instead of mmap. In Linux you can't map a non-regular file. Yes, you can read entire the file into the memory, but a size of the file can be very big and you will destroy the main advantage of mmap - the reading of only needed pieces of the file, not entire. And if you read it in in pieces, how can you be sure the file isn't changing on the system it's hosted on while you're looking at it? I beleive, mmap is not for distributed systems. -- Best regards, santucco
Re: [9fans] mmap
ron minnich wrote: On Tue, Jul 29, 2008 at 8:19 AM, erik quanstrom [EMAIL PROTECTED] wrote: you can't make the assumption that a file is local in *ix, either. in fact, for the last 20 years, every program run on a sunos/solaris machine has used mmap for the exec. mmap() is everywhere on Solaris. Even when you do something as trivial as sort(1) -- it'll try to mmap() the operands and only if that fails it is going to use read()/write(). Thanks, Roman. P.S. Which is not that bad of an optimization if you ask me...
Re: [9fans] mmap
As far as interfaces go, mmap() is pretty tragic - the underlying translation structures can express more interesting things, some of which are even worth doing. There have even been OSes that let userland apps play with their address spaces in far more interesting ways - KeyKOS and EROS come to mind. And they were even fast, or something. In a system like Plan 9, where your file servers are on the other side of a 9P link, this mmap thing seems dubious. If what you want is the convenience that you get from having all the bytes in memory, reading them all in wouldn't be too hard. mmap()s magic really arises when you have a page-cache-like-thing. --vs
Re: [9fans] mmap
As far as interfaces go, mmap() is pretty tragic - the underlying translation structures can express more interesting things, some of which are even worth doing. There have even been OSes that let userland apps play with their address spaces in far more interesting ways i think that's right, and that's the interesting case to investigate
Re: [9fans] mmap
i think that's right, and that's the interesting case to investigate provided, of course, that you're interested in the applications that might use it. otherwise it will just complicate things to no good effect.
[9fans] mmap
Mmap means many things to many people. Using mmap is most often not a matter of performance as much as it is a matter of flexibility: being able to mmap files is about as close as most operating systems get to exposing the underlying page table hardware, which lets applications that aren't happy with the OS-provided interface essentially design their own. The canonical example is databases, which would really prefer to be operating systems except for the whole having to write hardware drivers thing. 9vx is another example. If all you want is to map a few read-only (or at least non-write-back) files, like you'd need to implement dynamically-loaded shared libraries, that's not too hard to add. Back when I did the first cut of linuxemu, I changed segattach to accept 'file!a.out' as a segment name. If you've got a lot of these, you might need to change the segment array to be larger or perhaps dynamically sized. The particular interface I chose felt clumsy so I never even suggested putting it into the kernel, but mostly you can reuse the code that demand-loads ordinary Plan 9 binaries. The shared-library kind of mmap is by far the simplest mmap use case. Even allowing mmap as a surrogate for malloc would probably push you pretty far toward needing to rewrite the memory system, since the mmap-based mallocs tend to use a large number of small mappings. True mmap support in the Plan 9 kernel would require rewriting the entire memory system to work in terms of pages instead of the coarser regions the kernel calls segments (text, data, bss, stack). This would be a lot of work with almost zero benefit. I'd rather let those applications not happy with what Plan 9 provides just not use Plan 9. There's no need to be all things to all people. Also, on Plan 9, shared memory means that the memory at the time of rfork is shared. If one proc then does a segattach, the other proc sharing its memory will not see the segattach. The same is true of segdetach. This is another reason that you can't just reuse the existing mechanisms to hack in a traditional full-featured mmap. To really get those right you might need (gasp) TLB shootdowns, which would be even more work. It's nice that Plan 9's memory system is simple--just a few segments, each of which is a run of pages. True mmap support would destroy that. Please don't. Russ
Re: [9fans] mmap
in a traditional full-featured mmap i've noticed that some combinations of words are scarier than others. ☺ - erik
