Re: Perl 6 code - a possible compile, link, run cycle
Ingo Blechschmidt wrote: Hi, Yuval Kogman wrote: On Thu, Aug 25, 2005 at 15:42:28 +0200, Ingo Blechschmidt wrote: This section will contain all information needed: * User-defined operators * Other symbols exported by is export * Exported macros Okay, this raises a distinction: Compile time exports Runtime exports Well, all exports happen at compile-time, but you're right, some exports (regular subs) will probably not be used before runtime. Excuse my stupid question, but what about the equivalent Perl 6 case to the following. use Module qw{symbol}; BEGIN { die Not ready to compile if symbol(); }
Re: Perl 6 code - a possible compile, link, run cycle
On Thu, 25 Aug 2005 16:25:51 +0300, Yuval Kogman [EMAIL PROTECTED] wrote: [...] On Thu, Aug 25, 2005 at 11:16:56 -, David Formosa (aka ? the Platypus) = wrote: [...] use has the potentional to change the way the compiler parses the code. So use needs to be regarded. Hmm... Good point. I don't know how this is dealt with WRT to the every module is compiled with it's own compiler approach perl 6 is supposed to have. When compiling modules the compiler can seperate out stuff that will modify the caller's enviroment (exports ect) and thouse that will not. When use is seen, only this outward facing code needs to be examined. [...] Perhaps the use macro could do this on it's own. Seems sound. -- Please excuse my spelling as I suffer from agraphia. See http://dformosa.zeta.org.au/~dformosa/Spelling.html to find out more. Free the Memes.
Re: Perl 6 code - a possible compile, link, run cycle
On Wed, 24 Aug 2005 16:13:03 +0300, Yuval Kogman [EMAIL PROTECTED] wrote: [...] perl6 creates a new instance of the perl compiler (presumably an object). The compiler will only compile the actual file 'foo.pl', and disregard any 'require', 'use', or 'eval' statements. use has the potentional to change the way the compiler parses the code. So use needs to be regarded. -- Please excuse my spelling as I suffer from agraphia. See http://dformosa.zeta.org.au/~dformosa/Spelling.html to find out more. Free the Memes.
Re: Perl 6 code - a possible compile, link, run cycle
On Thu, Aug 25, 2005 at 11:16:56 -, David Formosa (aka ? the Platypus) wrote: On Wed, 24 Aug 2005 16:13:03 +0300, Yuval Kogman [EMAIL PROTECTED] wrote: [...] perl6 creates a new instance of the perl compiler (presumably an object). The compiler will only compile the actual file 'foo.pl', and disregard any 'require', 'use', or 'eval' statements. use has the potentional to change the way the compiler parses the code. So use needs to be regarded. Hmm... Good point. I don't know how this is dealt with WRT to the every module is compiled with it's own compiler approach perl 6 is supposed to have. Autrijus - do you have any solution? Either way, this simply implies that compilation is recursive, at least to determine whether the used module affects compilation. Perhaps the use macro could do this on it's own. I still think that regular linkage should be decoupled from compilation though, even if it's partially done during the compilation of a 'use' statement. -- () Yuval Kogman [EMAIL PROTECTED] 0xEBD27418 perl hacker /\ kung foo master: /me supports the ASCII Ribbon Campaign: neeyah!!! pgptwKRhiHhGC.pgp Description: PGP signature
Re: Perl 6 code - a possible compile, link, run cycle
On Aug 25, 2005, at 7:16 AM, David Formosa (aka ? the Platypus) wrote: On Wed, 24 Aug 2005 16:13:03 +0300, Yuval Kogman [EMAIL PROTECTED] wrote: [...] perl6 creates a new instance of the perl compiler (presumably an object). The compiler will only compile the actual file 'foo.pl', and disregard any 'require', 'use', or 'eval' statements. use has the potentional to change the way the compiler parses the code. So use needs to be regarded. Conceptually, Yuval's mechanism could still work. It's just that, when you go back and compile the 'use' lines, they may invalidate a compilation unit you thought you were finished with. --Dks
Re: Perl 6 code - a possible compile, link, run cycle
Hi, Yuval Kogman wrote: On Thu, Aug 25, 2005 at 11:16:56 -, David Formosa (aka ? the Platypus) wrote: On Wed, 24 Aug 2005 16:13:03 +0300, Yuval Kogman [EMAIL PROTECTED] wrote: perl6 creates a new instance of the perl compiler (presumably an object). The compiler will only compile the actual file 'foo.pl', and disregard any 'require', 'use', or 'eval' statements. use has the potentional to change the way the compiler parses the code. So use needs to be regarded. Hmm... Good point. I don't know how this is dealt with WRT to the every module is compiled with it's own compiler approach perl 6 is supposed to have. The indermediate form of a compiled .pm has to have a section containing information about the symbols exported by the module, similar to today's pilGlob section Pugs gives you if you use -CPIL, -CPerl5, or -CJSON: $ pugs -CPerl5 -we 'sub foo {...}' | \ perl -MYAML -we 'print Dump(eval )' --- !perl/PIL::Environment pilGlob: [...] - !perl/PSub pSubBody: PNil pSubLValue: 0 pSubName: 'main::foo' pSubParams: [...] pSubType: SubRoutine pilMain: !perl/PStmts pStmt: PNoop pStmts: PNil This section will contain all information needed: * User-defined operators * Other symbols exported by is export * Exported macros Note that *none* of these things influence the compilation of other .pls and .pms unless they're exported. I.e.: # Foo.pm module Foo { sub infix:+ ($a, $b) { 42 } say 1 + 1; # 42 } # Exported symbols: ::Foo # test.pl use Foo; say 1 + 1; # 2 # Bar.pm module Foo { sub infix:+ ($a, $b) is export(:DEFAULT) { 42 } say 1 + 1; # 42 } # Exported symbols: ::Foo, infix:+ # test.pl use Bar; say 1 + 1; # 42 --Ingo -- Linux, the choice of a GNU | We are Pentium of Borg. Division is futile. generation on a dual AMD | You will be approximated. Athlon!|
Re: Perl 6 code - a possible compile, link, run cycle
On Thu, Aug 25, 2005 at 15:42:28 +0200, Ingo Blechschmidt wrote: This section will contain all information needed: * User-defined operators * Other symbols exported by is export * Exported macros Okay, this raises a distinction: Compile time exports Runtime exports Modules can affect both the compilation of code that links against them by doing compile time exports like macros, infix subs, and so forth, and change the runtime by exporting functions and what not. When code is being compiled all the submodules are compiled, and the compile time exports are read from the linkable unit by the compiler, which determines which of these to use. This is used to affect compilation. The linkable units are cached. Any amount of time passes, and then the compiled code is run, triggerring linkage, which triggers compilation of modules. Since this compilation found the cached files, they are linked, and everyone is happy. There is one problem: If I compile foo.pl, which uses SomeModule, and SomeModule exports stuff at compile time, and foo.pl's object code is saved, what happens when SomeModule is upgraded? We can no longer link against it because potentially foo.pl's compilation depends on a certain state. The solution to this is to give the user some options: try to link the runtime symbols anyway, the user expects it to work try to recompile foo.pl if it's source code is available when compiling foo.pl, prelink SomeModule's runtime symbols into it, to ensure that no other version of SomeModule can affect foo.pl's emitted code. This can be a recursive or non recursive process. -- () Yuval Kogman [EMAIL PROTECTED] 0xEBD27418 perl hacker /\ kung foo master: /me kicks %s on the nose: neeyah! pgpCvMECA6CmW.pgp Description: PGP signature
Re: Perl 6 code - a possible compile, link, run cycle
Hi, Yuval Kogman wrote: On Thu, Aug 25, 2005 at 15:42:28 +0200, Ingo Blechschmidt wrote: This section will contain all information needed: * User-defined operators * Other symbols exported by is export * Exported macros Okay, this raises a distinction: Compile time exports Runtime exports Well, all exports happen at compile-time, but you're right, some exports (regular subs) will probably not be used before runtime. If I compile foo.pl, which uses SomeModule, and SomeModule exports stuff at compile time, and foo.pl's object code is saved, what happens when SomeModule is upgraded? Right. If you upgrade your glibc to an ABI-incompatible version, things will probably break. But, as you say, we can... try to link the runtime symbols anyway, the user expects it to work try to recompile foo.pl if it's source code is available when compiling foo.pl, prelink SomeModule's runtime symbols into it, to ensure that no other version of SomeModule can affect foo.pl's emitted code. This can be a recursive or non recursive process. --Ingo -- Linux, the choice of a GNU | Mr. Cole's Axiom: The sum of the generation on a dual AMD | intelligence on the planet is a constant; Athlon!| the population is growing.
Re: Perl 6 code - a possible compile, link, run cycle
On 8/25/05, Yuval Kogman [EMAIL PROTECTED] wrote: On Thu, Aug 25, 2005 at 11:16:56 -, David Formosa (aka ? the Platypus) wrote: On Wed, 24 Aug 2005 16:13:03 +0300, Yuval Kogman [EMAIL PROTECTED] wrote: [...] perl6 creates a new instance of the perl compiler (presumably an object). The compiler will only compile the actual file 'foo.pl', and disregard any 'require', 'use', or 'eval' statements. use has the potentional to change the way the compiler parses the code. So use needs to be regarded. Hmm... Good point. I don't know how this is dealt with WRT to the every module is compiled with it's own compiler approach perl 6 is supposed to have. It's pretty simple, really. If module A uses module B, then you go and compile B first. Then, when you get to use B in module A, you just call B::import at compile time. That is, the modules are compiled separately, but you're allowed to run things from an already compiled module while you are compiling another one. Luke
Perl 6 code - a possible compile, link, run cycle
WRT to PIL and compilation and all that, I think it's time to think about how the linker might look. As I see it the compilation chain with the user typing this in the prompt: perl6 foo.pl perl6 is a compiled perl 6 script that takes an input file, and compiles it, and then passes the compiled unit to the default runtime (e.g. parrot). perl6 creates a new instance of the perl compiler (presumably an object). The compiler will only compile the actual file 'foo.pl', and disregard any 'require', 'use', or 'eval' statements. The compiler produces an object representing a linkable unit, which will be discussed later. At this point the runtime kicks in. The runtime really runs compiled byte code for the runtime linker, which takes the compiled unit that the compiler emitted and prepares it for execution. The runtime linker checks if any inclusions of outside code have been made, and if so, invokes a search routine with the foreign module plugin responsible. For example 'use python:Numerical' will use the pyhon module plugin to produce a linkable unit. A given module will normally traverse a search path, find some source code, check to see if there is a valid cached version of the source code, and if needed, recompile the source code into another linkable unit. As the linker gets new linkable units it checks if they have any dependencies of their own, and eventually resolves all the data and code that modules take from one another. The resulting mess has only one requirement: that it can be run by the runtime - that is, byte code can be extracted out of it. If the modules expose more than just byte code with resolved dependencies in the modules, for example type annotations, serialized PIL, serialized perl 6 code, and so forth, it may, at this point, do any amount of static analysis as it pleases, recompiling, relinking, optimizing, inlining, performing early resolution (especially of MMD) and otherwise modifying code (provided it was asked to do this by the user). The optimization premise is: by the time it's linked it probably won't change too much. Link time isa magictime for resolving calls, inlining values, folding newly discovered constants, and so forth. Furthermore, a linker may cache the link between two modules, regardless of the calling script, so that optimization does not have to be repeated. The result is still the same: code that can be executed by the runtime. It just might be more efficient. The linker also must always be able to get the original version of the linked byte code back, either by reversing some changes, or keeping the original. At this point the runtime's runloop kicks in, starting at the start point in the byte code, and doing it's thing. Runtime loading of code (e.g. eval 'sub foo { }') simply reiterates the above procedure: 'sub foo { }' is compiled by the compiler, creating a linkable unit (that can give 'sub foo { }' to the world). The runtime linker gets a fault saying byte code state is changing, $compiled_code_from_eval is being ammended to $linked_byte_code_in_runtime_loop. The linker must then link the running code to the result of eval. To do this it may need to undo it's optimizations that assumed there was no sub foo. For example, if there is a call to 'foo()' somewhere in foo.pl, the linker may have just inlined a 'die no such sub foo()' error instead of the call. Another linker may have put in code to do a runtime search for the 'foo' symbol and apply it. The linker that did a runtime search that may fail doesn't need to change anything, but the linker which inlined a fatal error must undo that optimization now that things have changed. The behavior of the linker WRT to such things is dependant on the deployment setting. In a long running mod_perl application there may even be a linker that optimizes code as time goes by, slowly changing things to be more and more static. As the process progresses through time, the probability of new code being introduced is lower, so the CPU time is invested better. Furthermore, latency is not hindered, and startup is fast because the linker doesn't do any optimizations in the begining. This is part of the proposed optimizer chain, as brought up on p6l a month or so ago. Anyway, back to runtime linking. Once the code is consistent again, e.g. calls to foo() will now work as expected, eval gets the compiled code, and runs it. It just happens that 'sub foo { }' has no runtime effects, so evak returns, and normal execution is resumed. To get the semantics of 'perl -c' you force the linker to resolve everything, but don't actually go to the runloop. Linkable units are first class objects, and may be of a different class. This has merits when, for example, a linkable unit is implemented by an FFI wrapper. The FFI wrapper determines at link time what the foreign interface looks like, and then behaves like the linkable unit one might expect if it were a native interface. It can generate bytecode to call