Re: Tying Overloading
On Tue, Apr 24, 2001 at 01:18:49PM +0100, Nick Ing-Simmons wrote: What _really_ want to do is a dynamically scoped peep-hole optimize (actually a rewrite) of the op tree - written in perl. But I can't do that Yes, you can. Check out B::Generate. -- Britain has football hooligans, Germany has neo-Nazis, and France has farmers. -The Times
Re: So, we need a code name...
Andy Dougherty [EMAIL PROTECTED] writes: Starting with 'P' is useful so we can keep our acronyms such as PMC and PDD :-). It's just an internal code name, so it probably doesn't matter if it's taken for another purpose. PERIL (hi Tom!). -- Johan
Re: Tying Overloading
On Wed, 25 Apr 2001 11:01:07 -0300, Branden wrote: If the idea is supporting arbitrary add-on operators, which I believe will be done seldom, for only some specific classes, wouldn't it be better to have a ``catch all'' entry for operators different than the built-in ones? Of course, add-on operators would not have the same ``performance'' of built-in ones I think I second that. I would think of a fixed table for the built-in ones, and a linked list for the add-ons. It's not necessary that a new node is added for each and every method; instead, a structure similar to those used in TIFF files could be used, where each linked in node contains a table with several items, and a new node is only added when that table is full. -- Bart.
Re: wacko idea
At 11:28 AM 25/04/2001 -0400, Uri Guttman wrote: well, the real goal is persistancy, so how about this idea? instead of using Data::Dumper and cousins to stringify a structure, write it out and parse it back in, we provide a module which copies and converts a structure to an area and converts all of its pointers to offsets. then that can be written out in one binary chunk. reading it back in and converting back to real pointers will be much simpler and faster than any string parser... [snip] Well, the problem I see is when you have many different objects and you would like to have views of that system. For example, suppose object A has references to B and C, and that object D has references to B and C. If A, B and C are in the same AREA so that they can be serialized together as one, serializing D would be a problem, since probably A would go together with B and C or the AREA structure would have to be disassembled. I think we could use this to implement every structure (scalar, string, array, hash) in a contiguous memory area, using offsets to point to internal data, so that each separate object, like A, B or C, could be taked from memory and saved to a stream. Reading the stream and placing the read data into the memory, whatever position in memory it would be, would re-assemble the structure correctly. This could be done on a per-vtable base. For example, if we add SERIALIZE/DESERIALIZE entries on vtables, objects that implement their internal structure with contiguous allocated areas independent of initial position in memory, could implement SERIALIZE and DESERIALIZE trivially by only writing this memory area to a stream and by reading some bytes from a stream and placing it somewhere in the memory, all offsets would work as expected. OTOH, vtables that need to use split memory areas or that make reference to external objects, could have customized SERIALIZE and DESERIALIZE methods to handle their complexities and details. - Branden
Re: Tying Overloading
At 11:34 AM 25/04/2001 -0400, Dan Sugalski wrote: At 11:01 AM 4/25/2001 -0300, Branden wrote: 2) Anyway, even resizing vtables we would need some more indirection to determine in which position of the vtable is which operator. No. Each operator goes in a fixed position in the vtable, and it's the same for each table. Anything else is nasty, error prone, and slow. Well, isn't the idea supporting adding new operators on run-time? That cannot be done without having a table that maps the operator to the position the operator has in the table... If the distinction of compile-time and run-time matters, we have to remember that Perl allows run-time to occur inside compile-time (through BEGIN and END blocks) and also allows compile-time to occur inside run-time (through eval). Tables would potentially have to be updated while the program is running, because the change could be done inside of an eval. The other thing to consider is whether a vtable method is needed at all, or if what's really needed is another opcode. The opcode table will also be extendable and overridable, though it shares the same issues of thread safety as the vtables do. This guy is starting to scare me... (Though it's much easier to handle, ... [snip] And he continues to scare me... - Branden
Re: Tying Overloading
At 01:52 PM 25/04/2001 -0400, Dan Sugalski wrote: Seriously, I don't see why this should be a scary thing. So, the opcode table's extendable. So what? It'll make language X mode simpler, for some value of X, if that language can load in its own set of extended opcodes. Perhaps someone'll want to do functional programming with the Parrot runtime, and it makes the most sense from a speed and parser simplicity standpoint to treat certain activities as atomic things and emit a single bytecode for them. Extending the opcode table makes sense there. I agree, although I would argue that for functional programming you don't need anything but Perl: http://perl.plover.com/lambda/ (by MJD). Anyway, I would say opcode extendability should be analysed with a ROI optics, because I really don't know if having new opcodes (instead of faking them with subs -- if we have appropriate ways of locking synchronizing thread data) would be really necessary. - Branden
Re: Tying Overloading
At 02:01 PM 4/25/2001 -0300, Branden wrote: At 11:34 AM 25/04/2001 -0400, Dan Sugalski wrote: At 11:01 AM 4/25/2001 -0300, Branden wrote: 2) Anyway, even resizing vtables we would need some more indirection to determine in which position of the vtable is which operator. No. Each operator goes in a fixed position in the vtable, and it's the same for each table. Anything else is nasty, error prone, and slow. Well, isn't the idea supporting adding new operators on run-time? That cannot be done without having a table that maps the operator to the position the operator has in the table... Sure, and it's a table the parser keeps. By the time the bytecode is generated, the absolute vtable location will be known. If the distinction of compile-time and run-time matters, we have to remember that Perl allows run-time to occur inside compile-time (through BEGIN and END blocks) and also allows compile-time to occur inside run-time (through eval). Tables would potentially have to be updated while the program is running, because the change could be done inside of an eval. Yes, I'm aware of that. In the single-thread case it's pretty much irrelevant. We *know* the state of the interpreter, and we can mess with the vtables all we want, as long as offsets of known entries don't change. That's fine. (Though I, for one, wouldn't mind drawing a sharp line between compilation time and runtime, and forbidding a bunch of stuff at runtime (like messing with vtables) or compile time (like starting threads)) The other thing to consider is whether a vtable method is needed at all, or if what's really needed is another opcode. The opcode table will also be extendable and overridable, though it shares the same issues of thread safety as the vtables do. This guy is starting to scare me... Good, then my work is done. :) Seriously, I don't see why this should be a scary thing. So, the opcode table's extendable. So what? It'll make language X mode simpler, for some value of X, if that language can load in its own set of extended opcodes. Perhaps someone'll want to do functional programming with the Parrot runtime, and it makes the most sense from a speed and parser simplicity standpoint to treat certain activities as atomic things and emit a single bytecode for them. Extending the opcode table makes sense there. Think of it this way. Vtable extensions are for extensions to data access. Opcode extensions are more for extensions to control flow. Both make sense, but there's not really much overlap. And while you can do each with lots of wrapper code and no help from the parser or runtime, they become easier if that help is available. Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Tying Overloading
At 11:01 AM 4/25/2001 -0300, Branden wrote: At 09:39 AM 25/04/2001 -0400, Dan Sugalski wrote: At 06:46 PM 4/24/2001 -0700, Larry Wall wrote: I think we definitely have to be able to resize vtables at compile time, which is a form of run time. It's vaguely possible we could restrict multithreading during compile phase. We need to be single-threaded when we make any sort of wholesale structural change to the universe. It's a lot easier (and cheaper) to do it during a period where we're guaranteed to be single-threaded than it is to have all the running threads go stop and check periodically. (I don't really want to be coordinating on mutexes every statement or block or whatever) Are we really willing to pay the price and handle the complexities of *resizing* vtables? If the ROI is high enough, sure. Outside of the multithreaded issues, it's not all that tough. Some reallocs and a quick search-and-replace run over the PMC arenas. If the idea is supporting arbitrary add-on operators, which I believe will be done seldom, for only some specific classes, wouldn't it be better to have a ``catch all'' entry for operators different than the built-in ones? That would be an option, sure. It's slower, though, which is one of the things we're looking to avoid. Of course, add-on operators would not have the same ``performance'' of built-in ones, but: 1) We don't have to add an entry to *all* vtables for each operator we want to add to a *specific* class. We need at least enough entry to pitch a fit if the vtable method is called for a class. Though if we hide all the overflow in a single catchall entry that's easy enough to do. 2) Anyway, even resizing vtables we would need some more indirection to determine in which position of the vtable is which operator. No. Each operator goes in a fixed position in the vtable, and it's the same for each table. Anything else is nasty, error prone, and slow. 3) We wouldn't have to deal with the complexities of resizing vtables. 4) Having one only vtable entry as a ``catch all'' would allow defining new operators ``on the fly''. For example, I try to use the theta operator between two bignumbers. Even if theta wasn't defined, it would call the ``catch all'' of the bignumber vtable, that could do a search à la AutoLoader, or even define the operator based on the Unicode number of the symbol theta, or anything like that. The other thing to consider is whether a vtable method is needed at all, or if what's really needed is another opcode. The opcode table will also be extendable and overridable, though it shares the same issues of thread safety as the vtables do. (Though it's much easier to handle, as there's only one table that can be protected by a mutex, and the interpreters can certainly make sure that update access is properly synchronized, and we can also make sure we don't deploy any perl bytecode that requires the updated opcode table until the update is completed) Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: wacko idea
DS == Dan Sugalski [EMAIL PROTECTED] writes: DS At 06:37 PM 4/24/2001 -0500, David L. Nicol wrote: Uri Guttman wrote: i was looking at dan's PMC arena allocator struct and it reminded me of something an older language can do (which MJD has likened to an early perl :). ever heard of AREA in PL/I? it was a large chunk of ram dedicated to allocate memory from. what was special is that all pointers generated by these malloc calls (i forget the pl/1 syntax but it doesn't matter) were OFFSETS into the area and not direct pointers. the reason for this was that you can make a complex tree structure out of this AREA and then write the whole thing to disk as a single chunk, and in another program read it back in and your structure is back. since all the pointers are relative to the base of the AREA, you can read it back into any address in your virtual space. DS I'm not sure we'll do either. It's a neat idea, but at the moment DS things are multi-tiered enough that I'm not sure it'll work DS without enough extra effort that'll make a plain general-purpose DS serialization better. well, i am glad to see that some people actually read this amidst the . noise. there are many problems with my idea, particularly with syntax. in PL/I there is only 1 way to malloc storage so you can control the syntax easily and say this chunk/structure is coming from this area. IIRC you declared a special pointer BASED on the AREA and you did a malloc type call with the pointer and size as arguments and the pointer would be set with the address (offset from the AREA base, of course) of the new chunk. in perl, there are countless ways to allocate memory so forcing a syntax on all of those will be almost impossible. e.g. if an array is inside an area, it must be reallocated from there when resized and then all the offset pointers to it must be updated. maybe arrays/hashes/strings inside an area could be restricted from being resized? nah, nobody would like that. PL/I didn't do any of those things and you had to manage all the storage yourself. like i said it is a cool idea but probably too bizarre to directly implement in perl6. well, the real goal is persistancy, so how about this idea? instead of using Data::Dumper and cousins to stringify a structure, write it out and parse it back in, we provide a module which copies and converts a structure to an area and converts all of its pointers to offsets. then that can be written out in one binary chunk. reading it back in and converting back to real pointers will be much simpler and faster than any string parser. one issue is where do the offsets go? i ran into this problem with a large DB product which cast pointers to offsets to send data over a pipe and assumed 32 bits at all times then we had to port it to 64 bits. oy gevalt! but if we assume a C pointer can hold an offset and we are starting with with pointers, it could work. also this could use the GC traversal code to scan the structure when making the copy. all of that could be done in a module and not be part of the core. the data will not be portable between platforms but it will be reloadable anywhere in memory on the same architecture. any thoughts on that? uri -- Uri Guttman - [EMAIL PROTECTED] -- http://www.sysarch.com SYStems ARCHitecture and Stem Development -- http://www.stemsystems.com Learn Advanced Object Oriented Perl from Damian Conway - Boston, July 10-11 Class and Registration info: http://www.sysarch.com/perl/OOP_class.html
Re: Tying Overloading
At 03:08 PM 4/25/2001 -0300, Branden wrote: At 01:52 PM 25/04/2001 -0400, Dan Sugalski wrote: Seriously, I don't see why this should be a scary thing. So, the opcode table's extendable. So what? It'll make language X mode simpler, for some value of X, if that language can load in its own set of extended opcodes. Perhaps someone'll want to do functional programming with the Parrot runtime, and it makes the most sense from a speed and parser simplicity standpoint to treat certain activities as atomic things and emit a single bytecode for them. Extending the opcode table makes sense there. I agree, although I would argue that for functional programming you don't need anything but Perl Oh, sure, that's true and I'll give no argument. That doesn't mean that you can't do things to make something easier, however. You could, for example, argue that perl has no real need at the C level for opcodes with underlying support for arrays and hashes as data types, as you can emulate them with the scalar operations. Doesn't make them any less useful. Anyway, I would say opcode extendability should be analysed with a ROI optics, because I really don't know if having new opcodes (instead of faking them with subs -- if we have appropriate ways of locking synchronizing thread data) would be really necessary. Providing the capability is simple and straightforward, and will piggyback on top of the overridable opcode stuff. Besides, what makes you think we won't be doing at least some class of subs this way? :) Dan --it's like this--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
deferred vtable assignment?
Dan Sugalski wrote: 2) Anyway, even resizing vtables we would need some more indirection to determine in which position of the vtable is which operator. No. Each operator goes in a fixed position in the vtable, and it's the same for each table. Anything else is nasty, error prone, and slow. What if the decision in-vtable or not-in-vtable is deferred? The size of the vtable could be chosen late in the compilation. There could be hints. I am right now imagining vtable slots analogous to register entries for data in a C function. That we we also can deal with the aliasing of comparison operators to a variety of cmp/== (or not) on a case-by-case basis. if it is supposed to be an optimization, keep it an optimization, with a fall-back to a non-optimized paradigm. -- David Nicol 816.235.1187 [EMAIL PROTECTED] and they all say yodelahihu
