Re: RFC 155 - Remove geometric functions from core
At 07:37 PM 8/29/00 -0700, Russ Allbery wrote: Dan Sugalski [EMAIL PROTECTED] writes: On 29 Aug 2000, Russ Allbery wrote: I'd love to see Perl aggressively take advantage of new capabilities in dynamic loaders, though. Among other things, I'll point out that symbol versioning is the way that things like libc manage to be backward compatible while still changing things, and we should probably seriously consider using symbol versioning in a shared libperl.so as a means to provide that much desired and extremely difficult to implement stable API for modules and the XS-equivalent. This is where my lack of strange Unix knowledge comes to the fore. Is this really a problem? It seems to me to be a standard sort of thing to be dealing with. (OTOH, my platform of choice has 20-year-old executables as part of its test suite and a strong engineering bent, so I may be coming at things from a different direction than most folks) Well, it depends on what your goals are, basically. For most shared libraries, people don't take the trouble. That's OK--we will. :) Basically, no matter how well you design the API up front, if it's at all complex you'll discover that down the road you really want to *change* something, not just add something new (maybe just add a new parameter to a function). At that point, the standard Perl thing up until now to do is to just change it in a major release and require people to relink their modules against the newer version. And relink their applications that embed Perl. It's just hit me why VMS' system service interface has managed to handle this as well as it has over the years. Unfortunately one of the things that helped it's longevity is rather inconvenient for the average C programmer. I'll write up something more concrete once I've batted it around some in my brain, and we can see if I'm off-base or, if not, whether it's worth it. Not a big deal, and that's certainly doable. But it's possible to do more than that if you really want to. The glibc folks have decided to comment to nearly full binary compatibility for essentially forever; the theory is that upgrading libc should never break a working application even if the ABI changes. I'm not familiar with the exact details of how symbol versioning works, but as I understand it, this is what it lets you do. Both the old and the new symbol are available, and newly built applications use the new one while older applications continue to use the previous symbol. That means that all your older binary modules keep working, and if your applications that embed Perl are linked dynamically, you can even upgrade Perl underneath them without having to rebuild them. I'm not sure it's worth the trouble, but it's something to consider. I'm sure it is. I really, *really* want long-term binary compatibility. Luckily for us, perl may end up with a reasonably small external API, which'll make life easier. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: RFC 155 - Remove geometric functions from core
On Tue, 29 Aug 2000, Russ Allbery wrote: Not a big deal, and that's certainly doable. But it's possible to do more than that if you really want to. The glibc folks have decided to comment to nearly full binary compatibility for essentially forever; the theory is that upgrading libc should never break a working application even if the ABI changes. I'm not familiar with the exact details of how symbol versioning works, but as I understand it, this is what it lets you do. I'm sure the glibc folks indeed work very hard at this and are largely successful. I also know, however, that over the past couple of years or so, I've had to recompile nearly all of my applications on several occasions when I've upgraded glibc. Other times, glibc upgrades have gone without a hitch. It's probably my fault and probably somewhere deep in my personal library I'm incorrectly fiddling with stdio internals or something, but I just wanted to offer a counter data point that doing this sort of this robustly is, indeed, very hard. -- Andy Dougherty [EMAIL PROTECTED] Dept. of Physics Lafayette College, Easton PA 18042
Re: RFC 155 - Remove geometric functions from core
On Wed, 30 Aug 2000, Andy Dougherty wrote: On Tue, 29 Aug 2000, Russ Allbery wrote: Not a big deal, and that's certainly doable. But it's possible to do more than that if you really want to. The glibc folks have decided to comment to nearly full binary compatibility for essentially forever; the theory is that upgrading libc should never break a working application even if the ABI changes. I'm not familiar with the exact details of how symbol versioning works, but as I understand it, this is what it lets you do. I'm sure the glibc folks indeed work very hard at this and are largely successful. I also know, however, that over the past couple of years or so, I've had to recompile nearly all of my applications on several occasions when I've upgraded glibc. Other times, glibc upgrades have gone without a hitch. It's probably my fault and probably somewhere deep in my personal library I'm incorrectly fiddling with stdio internals or something, but I just wanted to offer a counter data point that doing this sort of this robustly is, indeed, very hard. I think we can pull this off if we're careful and draw really strict lines about what is and isn't public stuff. It's not easy, and it will mean we'll have to have some reference executables in a test suite for verification, but I think we can manage that. I do want to have a set of C/XS/whatever sources as part of the test suite as well--right now perl's test suite only tests the language, and I think we should also test the HLL interface we present, as it's just as important in some ways. dan
Re: RFC 155 - Remove geometric functions from core
On Wed, 30 Aug 2000, Dan Sugalski wrote: I think we can pull this off if we're careful and draw really strict lines about what is and isn't public stuff. It's not easy, and it will mean we'll have to have some reference executables in a test suite for verification, but I think we can manage that. I do want to have a set of C/XS/whatever sources as part of the test suite as well--right now perl's test suite only tests the language, and I think we should also test the HLL interface we present, as it's just as important in some ways. One of the off-the-cuff suggestions I sent in to one of the myriad polls for "fixing Perl", is that the bulk of Perl should be written in the XS replacement. This would force a two-pass build of perl, of course - the first pass to build the XSR, and the second to build perl itself. But the advantage was that it would really solidify the internals interface. And give you your test suite above. :-) -- Bryan C. Warnock ([EMAIL PROTECTED])
Re: RFC 155 - Remove geometric functions from core
Andy Dougherty [EMAIL PROTECTED] writes: I'm sure the glibc folks indeed work very hard at this and are largely successful. I also know, however, that over the past couple of years or so, I've had to recompile nearly all of my applications on several occasions when I've upgraded glibc. Other times, glibc upgrades have gone without a hitch. It's probably my fault and probably somewhere deep in my personal library I'm incorrectly fiddling with stdio internals or something, but I just wanted to offer a counter data point that doing this sort of this robustly is, indeed, very hard. It may not be your fault... my understanding is that glibc 2.0 really didn't do things right, and that glibc 2.1 did break some binary compatibility to fix some serious bugs. It's probably only fair to start holding glibc to this standard from 2.2 and up. Perl *should* have a *much* easier task than glibc, given that our interface is positively tiny compared to the entire C library. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/
Re: RFC 155 - Remove geometric functions from core
Well then. It is impossible to rearchitect it to make it shared text? Perhaps the first instance of perl sets up some vast shared memory segments and a way for the newcomers to link in to it and look at the modules that have been loaded, somewhere on this system, and use the common copy? This handwringing naysaying is depressing. Tom Christiansen wrote: Disastrously, you will then also lose the shared text component, which is what makes all this cheap when Perl loads. Since the modules will have to be pasted in the data segment of each process that wants them, they aren't going to be in a shared region, except perhaps for some of the non-perl parts of them on certain architectures. But certainly the Perl parts are *NEVER* shared. This sounds like a problem to be fixed. Relax, Tom, we'll take it from here. That's why the whole CGI.pm or IO::whatever.pm stuff hurts so badly: you run with 10 copies of Perl on your system (as many people do, if not much more than that), then you have to load them, from disk, into each process that wants them, and eth result of what you've loaded cannot be shared, since you loaded and compiled source code into non-shared parse trees. This is completely abysmal. Loading bytecode is no win: it's not shared text. --tom -- David Nicol 816.235.1187 [EMAIL PROTECTED] Ask me about sidewalk eggs
Re: RFC 155 - Remove geometric functions from core
On Tue, 29 Aug 2000, David L. Nicol wrote: Well then. It is impossible to rearchitect it to make it shared text? Perhaps the first instance of perl sets up some vast shared memory segments and a way for the newcomers to link in to it and look at the modules that have been loaded, somewhere on this system, and use the common copy? That approach invites big security problems. Any system that involves one program trusting another program to load executable code into their memory space is vulnerable to attack. This kind of thing works for forking daemons running identical code since the forked process trusts the parent process. In the general case of a second perl program starting on a machine, why would this second program trust the first program to not load a poison module? I don't believe you can simply "rearchitect it to make it shared text". This sounds like a problem to be fixed. Relax, Tom, we'll take it from here. Are you so sure? From where I'm sitting he's got some pretty tough points there. If you've got a solution then I'm quite suprised, which would be great. If not then I suggest you avoid writing the proverbial bad check. -sam
Re: RFC 155 - Remove geometric functions from core
Well then. It is impossible to rearchitect it to make it shared text? Perhaps the first instance of perl sets up some vast shared memory segments and a way for the newcomers to link in to it and look at the modules that have been loaded, somewhere on this system, and use the common copy? I'd be astonished to see a general-purpose, cross-platform, and maintainable solution to this problem. I predict that you'd at the very best, only address this a few places. Feel free to astonish me. This handwringing naysaying is depressing. Very well, then: I'll save it for an after-the-fact I-TOLD-YOU-SO, which, believe it or not, is truly *not* a pleasant thing to be able to say. --tom
Re: RFC 155 - Remove geometric functions from core
At 12:02 PM 8/29/00 -0600, Tom Christiansen wrote: Well then. It is impossible to rearchitect it to make it shared text? Perhaps the first instance of perl sets up some vast shared memory segments and a way for the newcomers to link in to it and look at the modules that have been loaded, somewhere on this system, and use the common copy? I'd be astonished to see a general-purpose, cross-platform, and maintainable solution to this problem. I predict that you'd at the very best, only address this a few places. Feel free to astonish me. It's possible we'll manage this with mmap()ing predigested bytecode, but I'm not entirely sure that's feasable--it means the optree (or whatever) that perl runs through would have to have no pointers at all in it, and that might be a rather big speed hit. (Presumably branches would need to do relative lookups and such) On the other hand, if we go the TIL route (which has its own cross-platform headaches) we could precompile segments of code into something that would be shareable as it'd be real executable code. This handwringing naysaying is depressing. Very well, then: I'll save it for an after-the-fact I-TOLD-YOU-SO, which, believe it or not, is truly *not* a pleasant thing to be able to say. Personally I'd rather have someone throwing wet blankets now, rather than later. If the ideas have merit enough to be worth doing then they'll survive the uncomfortable scrutiny. If they don't, better to get the ego bruising out of the way now, rather than after spending a month or more of wasted time. Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: RFC 155 - Remove geometric functions from core
Sam Tregar wrote: On Tue, 29 Aug 2000, David L. Nicol wrote: Well then. It is impossible to rearchitect it to make it shared text? Perhaps the first instance of perl sets up some vast shared memory segments and a way for the newcomers to link in to it and look at the modules that have been loaded, somewhere on this system, and use the common copy? That approach invites big security problems. Any system that involves one program trusting another program to load executable code into their memory space is vulnerable to attack. This kind of thing works for forking daemons running identical code since the forked process trusts the parent process. In the general case of a second perl program starting on a machine, why would this second program trust the first program to not load a poison module? does sysV shm not support the equivalent security as the file system? Did I not just describe how a .so or a DLL works currently? Yes, the later Perls would have to trust the first one to load the modules into the shared space correctly, and none of them would be allowed to barf on the couch. A paranoid mode would be required in which you don't use the shared pre-loaded module pool. In the ever-immenent vaporware implementation, this whole thing may be represented as a big file into which we can seek() to locate stuff. -- David Nicol 816.235.1187 [EMAIL PROTECTED] Yum, sidewalk eggs!
Re: RFC 155 - Remove geometric functions from core
Lightning flashed, thunder crashed and Tom Christiansen [EMAIL PROTECTED] m whispered: | Very well, then: I'll save it for an after-the-fact I-TOLD-YOU-SO, | which, believe it or not, is truly *not* a pleasant thing to be | able to say. Tom, we appreciate your constructive comments and your help in making sure we've considered all issues before we embark on a particular path. It is when you start calling us stupid for even suggesting that we look at the other paths that problems arise. There are lots of paths out there. Some of them may have quicksand in the middle of them, but can be safely navigated if we see them first. Some of them may be rabbit paths instead of super highways. Some of them may even lead to the pits of despair. But, if no one ever says "what about this path?" we'll never go anywhere except around the block. -spp
Re: RFC 155 - Remove geometric functions from core
On Tue, 29 Aug 2000, David L. Nicol wrote: does sysV shm not support the equivalent security as the file system? Well, no, I don't think it does. It supports permissions on individual segments but it doesn't support anything like directory perimssions. It might be enough, and it might not be. A user can run two programs and not expect one to have an automatic exploit on the other just because they're both Perl! Think "nobody". Yes, you'd provide a paranoid mode for experts to use to avoid the problems to which most users would be exposed. Great. Did I not just describe how a .so or a DLL works currently? Certainly not. You wrote only a few sentences. I'm no expert but I don't think that shared libraries are that simple. I also don't think they're implemented using SysV IPC shared memory, but you might know differently. In the ever-immenent vaporware implementation, this whole thing may be represented as a big file into which we can seek() to locate stuff. Zuh? What are you talking about? Is this some kind of Inline.pm-esque system? -sam
Re: RFC 155 - Remove geometric functions from core
David L . Nicol [EMAIL PROTECTED] writes:
does sysV shm not support the equivalent security as the file system?
mmap() has the file system.
Did I not just describe how a .so or a DLL works currently?
And behind the scenes that does something akin to:
int fd = open("file_of_posn_indepenant_byte_code",O_RDONLY);
struct stat st;
fstat(fd,st);
code_t *code = mmap(NULL,st.st_len,PROT_READ,MAP_SHARED,fd,0);
close(fd);
strace (linux) or truss (solaris) will show you what I mean.
And then trusts to OS to honour MAP_SHARED. (mmap() is POSIX.)
Win32 has "something similar" but I don't remember the function names off
hand.
Or you can embed your bytecode in
const char script[] = {...};
and link/dlopen() it and then you have classical shared text.
--
Nick Ing-Simmons
Re: RFC 155 - Remove geometric functions from core
At 07:32 PM 8/29/00 +, Nick Ing-Simmons wrote:
David L . Nicol [EMAIL PROTECTED] writes:
Did I not just describe how a .so or a DLL works currently?
And behind the scenes that does something akin to:
int fd = open("file_of_posn_indepenant_byte_code",O_RDONLY);
struct stat st;
fstat(fd,st);
code_t *code = mmap(NULL,st.st_len,PROT_READ,MAP_SHARED,fd,0);
close(fd);
Don't forget the fixup work that needs to be done afterwards. Loading the
library into memory's only the first part--after that the loader needs to
twiddle with transfer vectors and such so the unresolved calls into the
routines in the newly loaded library get resolved.
Dan
--"it's like this"---
Dan Sugalski even samurai
[EMAIL PROTECTED] have teddy bears and even
teddy bears get drunk
Re: RFC 155 - Remove geometric functions from core
On Tue, 29 Aug 2000, Nick Ing-Simmons wrote: David L . Nicol [EMAIL PROTECTED] writes: does sysV shm not support the equivalent security as the file system? mmap() has the file system. I wasn't aware that mmap() was part of SysV shared memory. My mistake? It's not on the SysV IPC man pages on my Linux system. The mmap manpage doesn't mention SysV IPC either. -sam
Re: RFC 155 - Remove geometric functions from core
Sam Tregar [EMAIL PROTECTED] writes: On Tue, 29 Aug 2000, Nick Ing-Simmons wrote: David L . Nicol [EMAIL PROTECTED] writes: does sysV shm not support the equivalent security as the file system? mmap() has the file system. I wasn't aware that mmap() was part of SysV shared memory. It is NOT. It is another (POSIX) way of getting shared memory bewteen processes. Even without MAP_SHARED OS will share un-modified pages between processes. It happens to be the way modern UNIX implemements "shared .text". i.e. the ".text" part of the object file is mmap()'ed into each process. My mistake? It's not on the SysV IPC man pages on my Linux system. The mmap manpage doesn't mention SysV IPC either. SysV IPC is a mess IMHO. My point was that if the "file system" is considered sufficient then mmap()ing file system objects will get you "shared code" or "shared data" without any tedious reinventing of wheels. -- Nick Ing-Simmons
Re: RFC 155 - Remove geometric functions from core
"ST" == Sam Tregar [EMAIL PROTECTED] writes: ST On Tue, 29 Aug 2000, Nick Ing-Simmons wrote: David L . Nicol [EMAIL PROTECTED] writes: does sysV shm not support the equivalent security as the file system? mmap() has the file system. ST I wasn't aware that mmap() was part of SysV shared memory. My ST mistake? It's not on the SysV IPC man pages on my Linux system. ST The mmap manpage doesn't mention SysV IPC either. mmap came from berkeley. i used it on early versions of sunos which was based on BSD. so calling it SysV IPC is wrong. uri -- Uri Guttman - [EMAIL PROTECTED] -- http://www.sysarch.com SYStems ARCHitecture, Software Engineering, Perl, Internet, UNIX Consulting The Perl Books Page --- http://www.sysarch.com/cgi-bin/perl_books The Best Search Engine on the Net -- http://www.northernlight.com
Re: RFC 155 - Remove geometric functions from core
On Tue, Aug 29, 2000 at 06:43:34PM -0400, Uri Guttman wrote: "ST" == Sam Tregar [EMAIL PROTECTED] writes: ST On Tue, 29 Aug 2000, Nick Ing-Simmons wrote: David L . Nicol [EMAIL PROTECTED] writes: does sysV shm not support the equivalent security as the file system? mmap() has the file system. ST I wasn't aware that mmap() was part of SysV shared memory. My ST mistake? It's not on the SysV IPC man pages on my Linux system. ST The mmap manpage doesn't mention SysV IPC either. mmap came from berkeley. i used it on early versions of sunos which was based on BSD. so calling it SysV IPC is wrong. Yup. I think somebody said that mmap() is POSIX. It isn't. POSIX realtime extensions (and Single UNIX Spec) have shared memory objects, which are different from either SysV IPC or mmap(). The smos have a system-wide flat namespace, not connected to the usual filesystem namespace (in the definition, that is, nobody of course forbids making it visible) mmap() is also in the SUS. Executive summary: there are three different "shared memory" APIs. -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: RFC 155 - Remove geometric functions from core
"TC" == Tom Christiansen [EMAIL PROTECTED] writes: Yup. I think somebody said that mmap() is POSIX. It isn't. TC Are you sure? TC http://slacvx.slac.stanford.edu/HELP/POSIX/CALLABLE_FUNCTIONS/MMAP TCThe mmap() function maps process addresses to a memory object. i think he meant POSIX didn't create the mmap call (which we agree was a bsd thing first). jarkko already said that posix has both mmap and their own shared memory api. uri -- Uri Guttman - [EMAIL PROTECTED] -- http://www.sysarch.com SYStems ARCHitecture, Software Engineering, Perl, Internet, UNIX Consulting The Perl Books Page --- http://www.sysarch.com/cgi-bin/perl_books The Best Search Engine on the Net -- http://www.northernlight.com
Re: RFC 155 - Remove geometric functions from core
David L Nicol [EMAIL PROTECTED] writes: This is what I was talking about when I suggested the language maintain a big list of all the addresses of each function, and after the function gets loaded or compiled it is added to the big list, and after this stage the placeholder in the op can be replaced with a longjump. Since the shared segments live at different addresses in different processes (or should I have stayed awake through that lecture) I'm not sure I'm completely following what you're arguing for here, but be careful not to go too far down the road of duplicating what the dynamic loader already knows how to do. There be dragons; that stuff is seriously baroque. You really don't want to reimplement it. I'd love to see Perl aggressively take advantage of new capabilities in dynamic loaders, though. Among other things, I'll point out that symbol versioning is the way that things like libc manage to be backward compatible while still changing things, and we should probably seriously consider using symbol versioning in a shared libperl.so as a means to provide that much desired and extremely difficult to implement stable API for modules and the XS-equivalent. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/
Re: RFC 155 - Remove geometric functions from core
On 29 Aug 2000, Russ Allbery wrote: I'm not sure I'm completely following what you're arguing for here, but be careful not to go too far down the road of duplicating what the dynamic loader already knows how to do. There be dragons; that stuff is seriously baroque. You really don't want to reimplement it. I'd very much like to not do it. It's bad enough to have to do that sort of thing with perl code we completely control--having to do it portably for what's essentially native code is more work than I'd really like to do. Clever People have already provided the capabilities on individual platforms and that's just fine by me. I'd love to see Perl aggressively take advantage of new capabilities in dynamic loaders, though. Among other things, I'll point out that symbol versioning is the way that things like libc manage to be backward compatible while still changing things, and we should probably seriously consider using symbol versioning in a shared libperl.so as a means to provide that much desired and extremely difficult to implement stable API for modules and the XS-equivalent. This is where my lack of strange Unix knowledge comes to the fore. Is this really a problem? It seems to me to be a standard sort of thing to be dealing with. (OTOH, my platform of choice has 20-year-old executables as part of its test suite and a strong engineering bent, so I may be coming at things from a different direction than most folks) I've been working on a spec for the API that'll hopefully isolate extensions enough that we can yank the guts around at will without affecting them. I'm really hoping that an extension built against perl 6.0.1 (presumably the first stable release... :) will work against perl 6.12.4. That's the plan, at least. Dan
Re: RFC 155 - Remove geometric functions from core
Dan Sugalski [EMAIL PROTECTED] writes: On 29 Aug 2000, Russ Allbery wrote: I'd love to see Perl aggressively take advantage of new capabilities in dynamic loaders, though. Among other things, I'll point out that symbol versioning is the way that things like libc manage to be backward compatible while still changing things, and we should probably seriously consider using symbol versioning in a shared libperl.so as a means to provide that much desired and extremely difficult to implement stable API for modules and the XS-equivalent. This is where my lack of strange Unix knowledge comes to the fore. Is this really a problem? It seems to me to be a standard sort of thing to be dealing with. (OTOH, my platform of choice has 20-year-old executables as part of its test suite and a strong engineering bent, so I may be coming at things from a different direction than most folks) Well, it depends on what your goals are, basically. For most shared libraries, people don't take the trouble. Basically, no matter how well you design the API up front, if it's at all complex you'll discover that down the road you really want to *change* something, not just add something new (maybe just add a new parameter to a function). At that point, the standard Perl thing up until now to do is to just change it in a major release and require people to relink their modules against the newer version. And relink their applications that embed Perl. Not a big deal, and that's certainly doable. But it's possible to do more than that if you really want to. The glibc folks have decided to comment to nearly full binary compatibility for essentially forever; the theory is that upgrading libc should never break a working application even if the ABI changes. I'm not familiar with the exact details of how symbol versioning works, but as I understand it, this is what it lets you do. Both the old and the new symbol are available, and newly built applications use the new one while older applications continue to use the previous symbol. That means that all your older binary modules keep working, and if your applications that embed Perl are linked dynamically, you can even upgrade Perl underneath them without having to rebuild them. I'm not sure it's worth the trouble, but it's something to consider. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/
Re: RFC 155 - Remove geometric functions from core
Jarkko Hietaniemi [EMAIL PROTECTED] writes: bytes microperl, which has almost nothing os dependent (*) in it 1212416 shared libperl 1277952 bytes + perl 32768 bytes1310720 dynamically linked perl1376256 statically linked perl with all the core extensions2129920 (*) I haven't tried building it in non-UNIX boxes, so I can't be certain of how fastidiously features have been disabled. "bytes" of what? - size of executable, size of .text, ??? If we are taling executable with -g size then a lot of that is symbol-table and is tedious repetition of "sv.h" co. re-itteerated in each .o file. But the basic point is that these things are small. So ripping all this 'cruft' would save us about 100-160 kB, still leaving us with well over a 1MB-plus executable. It's Perl itself that's big, not the thin glue to the system functions. My support for the idea is not to reduce the size of perl in the UNIX case, but to allow replacement. I would also like to have the mechanism worked out and "proven" on something that we know gets used so that we can have good solid testing of the mechanism. Then something less obvious (say Damian's any/all operators) which might be major extra size and not of universal appeal can use a well tried mechanism, and we can flip default to re-link sockets or sin/cos/tan into the core. -- Nick Ing-Simmons
Re: RFC 155 - Remove geometric functions from core
On Sun, Aug 27, 2000 at 08:37:38PM +, Nick Ing-Simmons wrote: Jarkko Hietaniemi [EMAIL PROTECTED] writes: bytes microperl, which has almost nothing os dependent (*) in it 1212416 shared libperl 1277952 bytes + perl 32768 bytes 1310720 dynamically linked perl 1376256 statically linked perl with all the core extensions 2129920 (*) I haven't tried building it in non-UNIX boxes, so I can't be certain of how fastidiously features have been disabled. "bytes" of what? - size of executable, size of .text, ??? Sizes of the executables/libs, code fully optimized -- and symbols stripped. If we are taling executable with -g size then a lot of that is symbol-table and is tedious repetition of "sv.h" co. re-itteerated in each .o file. But the basic point is that these things are small. -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: RFC 155 - Remove geometric functions from core
Lightning flashed, thunder crashed and Steve Fink [EMAIL PROTECTED] whispered: | Depends on your definition of "module". Many people seem to be assuming | "module" eq "shared library". Yes, exactly. I use module as a generic term for something other than the main perl binary itself, a black box if you will. The more modular we can make perl, I think the easier it is for it to be ported or embedded or whatever. It is very easy to simply not install a "module" or write stub functions, or even to add a new function. It is not so easy to remove things from "the core". Note some of the deprecated features we've lived with for years (such as EQ and friends). -spp
Re: RFC 155 - Remove geometric functions from core
At 09:12 AM 8/25/00 -0400, Stephen P. Potter wrote: As you say, 200 lines isn't much. But combine that with the IPC, the environment, the system, etc it all adds up. Not to much, though. We've been down this road for perl 5. You'd be surprised at how little code gets removed if you yank most of the functions under discussion. (They're generally trivial wrappers around library calls, with very little code involved) Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: RFC 155 - Remove geometric functions from core
That's the basic goal behind my RFCs for moving things to modules. In general, I hope to make the language cleaner, easier to learn and use, and easier to extend. "Clean"? What is "clean"? Huh? And since when has Perl ever been supposed to be "clean"? I've got plenty of quotage from Larry to the contrary. You must now define objective and quantifiable criteria for what does and thus what does not constitute: * Making a language cleaner * Making a language easier to learn * Making a language easier to use * Making a language easier to extend Provide working frameworks for those in concrete terms that can be applied to your proposals in a non-feel-good fashion, one that is discretely measurable by any party, and only then we can see whether they actually make any sense. Right now, they seem like random fantasies without any basis in reality. Note that it is more important that a language be easier to use than anything else you've listed. If at the same time the language became better performing because of a removal of some of the core, all the better. As you say, 200 lines isn't much. But combine that with the IPC, the environment, the system, etc it all adds up. What the bloody blazes are you going to do with a language that can't do systems work? Number crunching, I suppose. Oh wait--you already stripped that out, too. :-( Remember: big languages make small programs, and small language make big programs. Larry in observing this has clearly weighed in on the side of small programs. Micro-language people always have forth. --tom
Re: RFC 155 - Remove geometric functions from core
Not to much, though. We've been down this road for perl 5. You'd be surprised at how little code gets removed if you yank most of the functions under discussion. (They're generally trivial wrappers around library calls, with very little code involved) Thaniks -- I wish people wouldn't forget this. And it seems crazy to worry about sacrificing convenient functionality by saving a few bytes when memory is as incredibly cheap as it is. --tom
Re: RFC 155 - Remove geometric functions from core
Dan wrote: At 09:12 AM 8/25/00 -0400, Stephen P. Potter wrote: As you say, 200 lines isn't much. But combine that with the IPC, the environment, the system, etc it all adds up. Not to much, though. We've been down this road for perl 5. You'd be surprised at how little code gets removed if you yank most of the functions under discussion. (They're generally trivial wrappers around library calls, with very little code involved) so is all this concern about the size of perl due to the hope of making perl "embeddable"? with 128 meg of ram and a 40 gig drive, I'll assume its not due to system limitations. Greg London
Re: RFC 155 - Remove geometric functions from core
Or, more succinctly, we're not going to screw with perl without a *darned* good reason. This is the most beautiful thing I've read in days. --tom
Re: RFC 155 - Remove geometric functions from core
Dan Sugalski writes: : Or, more succinctly, we're not going to screw with perl without a *darned* : good reason. Er, perl is already screwed--it's Perl we're trying to preserve and grow. Larry
Re: RFC 155 - Remove geometric functions from core
Dan Sugalski wrote: Or, more succinctly, we're not going to screw with perl without a *darned* good reason. Well, in the big picture, we *are* screwing with perl, and we know that we have darned good reason to. But to apply this metric to every fine-grained aspect of perl5 is silly. Perl6 should be a well-designed, consistent, integrated opus. If it goes against the grain of perl6's design to have system-specific system calls built in, then they shouldn't be built in, regardless of how it was in perl5. -- John Porter We're building the house of the future together.
Re: RFC 155 - Remove geometric functions from core
Tom Christiansen writes: : Or, more succinctly, we're not going to screw with perl without a *darned* : good reason. : : This is the most beautiful thing I've read in days. Bear in mind there are lots of darned good reasons. :-) Larry
Re: RFC 155 - Remove geometric functions from core
Larry Wall wrote: Dan Sugalski writes: : Or, more succinctly, we're not going to screw with perl without a *darned* : good reason. Er, perl is already screwed--it's Perl we're trying to preserve and grow. Man; leave it to Larry to say the right thing, in the best way. :-) -- John Porter
Re: RFC 155 - Remove geometric functions from core
But to apply this metric to every fine-grained aspect of perl5 is silly. Perl6 should be a well-designed, consistent, integrated opus. If it goes against the grain of perl6's design to have system-specific system calls built in, then they shouldn't be built in, regardless of how it was in perl5. That's silly. *ALL* syscalls are system specific. This is starting to sound like Java, which has its own brand of revisionism. You'll take my fork when you pry it out of my cold, dead hands. --tom
Re: RFC 155 - Remove geometric functions from core
Tom Christiansen wrote: But to apply this metric to every fine-grained aspect of perl5 is silly. Perl6 should be a well-designed, consistent, integrated opus. If it goes against the grain of perl6's design to have system-specific system calls built in, then they shouldn't be built in, regardless of how it was in perl5. That's silly. *ALL* syscalls are system specific. Well, duh. So what? Did you really miss my point by so many furlongs? You'll take my fork when you pry it out of my cold, dead hands. Perl will ship with fork. But maybe in a loadable library, eh? -- John Porter We're building the house of the future together.
Re: RFC 155 - Remove geometric functions from core
Well, duh. So what? Did you really miss my point by so many furlongs? Whatever. You'll take my fork when you pry it out of my cold, dead hands. Perl will ship with fork. But maybe in a loadable library, eh? More of this nonsense, eh? I just fail to understand the urge to eviscerate. Why don't we just say that Perl isn't for systems work anymore, and remove everything that diddles $!, or $?, or anything that might call anything from the C library. Useless. --tom
Re: RFC 155 - Remove geometric functions from core
You don't beneficially make Perl any more anything by ghettoizing its useful systems functionality. You certainly don't make it more "portable". I still can't use any of my myriad useful systems hackery script if I try to run them on CP/M, and you aren't going to change that one whit. So what if it's in a ghetto hole? It still doesn't work. --tom
Re: RFC 155 - Remove geometric functions from core
Tom Christiansen wrote: Why don't we just say that Perl isn't for systems work anymore, and remove everything that diddles $!, or $?, or anything that might call anything from the C library. As in "remove mountains", yes. -- John Porter We're building the house of the future together.
Re: RFC 155 - Remove geometric functions from core
If the behind-the-scenes shenanigans are invisible and performance is at worst acceptable, does it really matter how things are implemented? So long as you can still say things like perl -le 'print scalar getpwuid(1)' and that script still continue to work unaltered, then I suppose not--assuming no seriously grave performance issues. --tom
Re: RFC 155 - Remove geometric functions from core
Tom Christiansen writes: : More of this nonsense, eh? Please don't use fighting words in here. : I just fail to understand the urge to eviscerate. Why don't we just : say that Perl isn't for systems work anymore, and remove everything : that diddles $!, : or $?, or anything that might call anything from : the C library. This is confusing Perl and perl again. This is not terribly useful, especially in -internals. The degree to which Perl 6 (the language) will be useful for systems work has almost nothing to do with the method by which perl6 (the program) implements the given functionality. In fact, we're trying to *decouple* those ideas further so that we can more easily add such functionality to Perl without forcing people to read miles of camel entrails before they begin. If we can make it run faster in the process, that'll be even better. We're redesigning everything. And we'll do the best job we can with the brains we've got. We will bear people's fears in mind, but we will not be controlled by such fears. Rather, we will try to give fear its proper weight--which is neither too much, nor too little. Fasten your seat belts, folks. Larry
Re: RFC 155 - Remove geometric functions from core
At 11:36 AM 8/25/00 -0600, Tom Christiansen wrote: If the behind-the-scenes shenanigans are invisible and performance is at worst acceptable, does it really matter how things are implemented? So long as you can still say things like perl -le 'print scalar getpwuid(1)' and that script still continue to work unaltered, then I suppose not--assuming no seriously grave performance issues. That's one of the working goals... Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: RFC 155 - Remove geometric functions from core
Larry Wall writes: Tom Christiansen writes: : More of this nonsense, eh? Please don't use fighting words in here. On the subject of fighting words, I owe everyone an apology for my language on the subject of Perl being the only thing that can parse Perl. I've been banging my head against program filters lately, and I'm frustrated that it's so hard. That's no excuse for exposing you all to my frustration. Sorry. So please take the suggestion (that Perl be more easily toked) without the swear words. Thanks. Nat
Re: RFC 155 - Remove geometric functions from core
On Fri, Aug 25, 2000 at 09:12:32AM -0400, Stephen P. Potter wrote:
Lightning flashed, thunder crashed and Michael G Schwern [EMAIL PROTECTED] wh
ispered:
| PS The idea of adding acos, asin and tan is good.
You just answered your own question. It is very difficult to add new
functions to the core. It is very easy to write new modules. Doesn't it
make sense that if you have to use Math::Trig to get to acos and friends,
you might as well make the language definition clean and say all of acos
friends should be in that module, not some in the core?
Actually I was suggesting that acos, asin and tan be added to the core.
Most likely, splitting this out into a module wouldn't make anything
much simpler. To get anything like the performance math functions
need, Math::Trig could not be written in plain Perl. It would have to
be written in C or XS (or whatever perl6 winds up being written in)
probably calling the system's math libraries and still dragging in all
the basic problems of a core patch and configuration.
That's the basic goal behind my RFCs for moving things to modules. In
general, I hope to make the language cleaner, easier to learn and use, and
easier to extend. If at the same time the language became better
performing because of a removal of some of the core, all the better. As
you say, 200 lines isn't much. But combine that with the IPC, the
environment, the system, etc it all adds up.
I think I basically agree with tchrist here. This is nickel and dime
stuff. Cutting out the math functions isn't going to do squat. IPC
and the system... that might do something, but Dan pointed out that it
doesn't amount to much either.
I think we should back up and reconsider what the intent of this RFC
is.
If the intent is to make perl more maintainable and easier to patch,
I've already commented on that. You're just shuffling code around.
And only a small fraction at that.
If you wish to make perl smaller and faster, just tearing things out
isn't going to help. Its hit-or-miss optimizing. You could remove
half the core functions and find out you only gained 5%.
Like all other optimizing attempts, the first step is analysis.
People have to sit down and systematically go through and find out
what parts of perl (and Perl) are eating up space and speed. The
results will be very surprising, I'm sure, but it will give us a
concrete idea of what we can do to really help out perl's performance.
There should probably be an RFC to this effect, and I'm just visiting
here in perl6-language so I dump it on somebody else.
exit('stage' 1);
--
Michael G Schwern http://www.pobox.com/~schwern/ [EMAIL PROTECTED]
Just Another Stupid Consultant Perl6 Kwalitee Ashuranse
When faced with desperate circumstances, we must adapt.
- Seven of Nine
Re: RFC 155 - Remove geometric functions from core
On Fri, Aug 25, 2000 at 09:20:53AM -0400, Dan Sugalski wrote: At 09:12 AM 8/25/00 -0400, Stephen P. Potter wrote: As you say, 200 lines isn't much. But combine that with the IPC, the environment, the system, etc it all adds up. Not to much, though. We've been down this road for perl 5. You'd be surprised at how little code gets removed if you yank most of the functions under discussion. (They're generally trivial wrappers around library calls, with very little code involved) Here are some numbers for people who have forgotten the above. Using the latest perl development sources, using an unnamed UNIX: bytes microperl, which has almost nothing os dependent (*) in it 1212416 shared libperl 1277952 bytes + perl 32768 bytes 1310720 dynamically linked perl 1376256 statically linked perl with all the core extensions 2129920 (*) I haven't tried building it in non-UNIX boxes, so I can't be certain of how fastidiously features have been disabled. So ripping all this 'cruft' would save us about 100-160 kB, still leaving us with well over a 1MB-plus executable. It's Perl itself that's big, not the thin glue to the system functions. -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
Re: RFC 155 - Remove geometric functions from core
Lightning flashed, thunder crashed and Tom Christiansen [EMAIL PROTECTED] m whispered: | More of this nonsense, eh? I just fail to understand the urge | to eviscerate. Why don't we just say that Perl isn't for | systems work anymore, and remove everything that diddles $!, | or $?, or anything that might call anything from the C library. Ok, here's my new RFC. This should handle all of Tom's objections: =head1 TITLE Perl is Tom's private domain. =head1 VERSION Maintainer: Stephen P. Potter [EMAIL PROTECTED] Date: 8/25/2000 Version: 1 Mailing List: [EMAIL PROTECTED] =head1 ABSTRACT (Dredging up really old jokes) Tom is a highly developed artificial intelligence, not a person like everyone thinks. Tom is also completely perfect. Therefore, we should take everything that Tom says as gospel. =head1 DESCRIPTION It is readily apparent that no one, except Tom, has any good ideas on what should and should not happen to perl. Any change that anyone else suggests must be because either they are too stupid to understand what Perl (or perl) is, because they want a different language, or is designed strictly to make Tom's life miserable. As such, we should all just abandon the entire idea of perl6, unless Tom suggests a change. Every RFC which has been submitted so far (unless Tom submitted it) should be dropped. =head1 IMPLEMENTATION There is no implementation. Tom hasn't said we are allowed to change anything yet. Everything is perfect. =head1 REFERENCES See any posting of Tom's, especially if it is an answer to someone else's post.
Re: RFC 155 - Remove geometric functions from core
Your coquettish plot to reveal the desperate yearning of your nethermost alimentary canal for multiply redundant new egresses is neither charming nor subtle--nor even innovative. Although it would be no great trouble for me to don the vestments and sharpen the cutlery to serve as haruspex for so joyous an event as you have requested, I must confess doubt that these divinations might augur anything that would prove to our current forum useful. For the sake of general decorum, I must therefore deny your ardent request of public evisceration, and instead quietly counsel you to maintain discreetly hidden from public display the steaming faeculae that your mouth has just now wetly ejected like some hydrophobic and coprophagous cur subjected to explosive decompression. --tom
Re: RFC 155 - Remove geometric functions from core
Stephen P. Potter writes: =head1 TITLE Perl is Tom's private domain. That's unproductive. Nat
Re: RFC 155 - Remove geometric functions from core
And that's offensive. What's offensive to one person is amusing to the next. Please act like a grown-up. Stephen cast the first stone, but that's no excuse for you to reply with a boulder. Sure it is: when a hoodlum jumps you with a knife, there's no reason to roll over and quietly submit to the death of a thousand cuts. No, you pull an Indy by responding with overwhelming firepower to dispatch the cretin forthwith before he gets cocky. Otherwise you're a willing victim with an FMH sign on his butt waiting for further abuse as the next bandit decides to chew on you. As nobody else said mum about that scat, I took care of it myself. (hint: grown-ups would apologise at this point) I'm sorry he said that. --tom
Re: RFC 155 - Remove geometric functions from core
Tom Christiansen writes: : Please act like a grown-up. Stephen cast the : first stone, but that's no excuse for you to reply with a boulder. : : Sure it is: when a hoodlum jumps you with a knife, there's no reason : to roll over and quietly submit to the death of a thousand cuts. : No, you pull an Indy by responding with overwhelming firepower to : dispatch the cretin forthwith before he gets cocky. Otherwise : you're a willing victim with an FMH sign on his butt waiting for : further abuse as the next bandit decides to chew on you. As nobody : else said mum about that scat, I took care of it myself. I despise escalation. You seem to enjoy it. Larry
Re: RFC 155 - Remove geometric functions from core
Tom Christiansen writes: further abuse as the next bandit decides to chew on you. As nobody else said mum about that scat, I took care of it myself. 90 minutes passed on a Friday night. That doesn't mean it wasn't going to be dealt with. (hint: grown-ups would apologise at this point) I'm sorry he said that. Cute, but flippancy doesn't convince me you're capable of maturity. If you can't behave, I'll treat you like the little children you are and give you a time out. If that fails to moderate your behaviour, I will take the ability to make trouble out of your grasp by preventing the mailing list software from accepting messages from you. This goes for everyone, Stephen as well as Tom. Name-calling and petulant messages are the kind of things we're trying to leave behind. This is supposed to be about perl6, not personalities. It makes me grumpy that this is distracting from the work we're supposed to be doing. Please don't waste any more of our time with this idiocy. Thank you, Nat
Re: RFC 155 - Remove geometric functions from core
Tom Christiansen wrote: So ripping all this 'cruft' would save us about 100-160 kB, still leaving us with well over a 1MB-plus executable. It's Perl itself that's big, not the thin glue to the system functions. Moreover, if you rip out that 100-160 kb, allocating it to modules, then I can guarantee you that it will cost significantly more memory as a module to pull in that if it here already there. There's always some overhead involved in these things. Notice how the Byteloader produces much bigger executables than the originals. Some work has been done to fix that, but as a minimum, you've got the Byteloader module itself. And it gets worse... Depends on your definition of "module". Many people seem to be assuming "module" eq "shared library". I submit that a better definition would be anything that uses the Perl6 Core API. The vision I have is that all existing builtins _could_ be linked with the "perl core". The new builtins (a small list) always will be -- they require inside knowledge of data structures that should specifically not be exposed to extensions. (With a "please", not a shotgun.) Any or all of the "core modules" could be placed into one or separate shared libraries. They would have no more access to the internals than any 3rd party module. (That's my real definition of a module -- it's a property of the architecture, not of the linkage, preprocessing, author, or whatever else.) What this means is that we have to define a recommended API for modules to use (and warn that compatibility is not assured if they dig deeper than this), and to make sure that sin(), socket() etc. strictly obey this discipline. If this API is too raw for the convenience of many extensions, then we can always layer it with a kindler gentler API, but the bottom-most API should be the important one. At compile time, you can choose which things you want linked into the perl core, which ones to stick in a shared library, and which ones to leave out entirely, with a warning "what you are about to compile is not Perl" if they choose to leave out any "core module" entirely. In fact, the binary should then be called "pht" rather than "perl". Whether this is implemented via the preprocessor, autogenerated code, or ? is a separate issue. It seems easily possible to define an API that allows either static or dynamic linking with no performance loss. Maybe this is what some people are already assuming. I just thought I should spell it out. It's probably implied by the RFC that says user subs should be indistinguishable from opcodes (doesn't one that sound something like that exist? I'm too lazy to go look.)
