Perl grammar for Perl5 - Perl6
Dear Perl6 Language, I am Perl user from near year 0. For me the easiest way to learn (, track, and get to the point of contributing to) Perl6 would be a Perl grammar (a regex rule set in, preferably, Perl6) that transforms any Perl5 script into a Perl6. Of couse, besides learning Perl6 for a regex'r or Perl5'r such as myself, and tracking, and documenting 6, it would have huge use for Perl5 users making or considering the transition. For example one can infer the structure and some of the rules from http://svn.openfoundry.org/pugs/docs/other/porting_howto which is however written out in informal (not regex) rules such as $array[idx] - @array[idx] Is there such a Perl5-Perl6 translator underway? Sincerely, Peter Schwenn www.schwenn.com p.s. I'm not yet up to originating a good one, but believe I could make contributions in extending and checking it. p.s. A Perl5 starting point might be better since running it on itself would provide and initiate at least 2 major test modes. p.s. The developing form of such a grammar could likely lead to a grammar package which facilitates rule sets for languages in other domains, in terms of illuminating means of choosing among modes for rule ordering, collecting, scoping, re-application, recursion, exclusion and so forth.
Re: Perl grammar for Perl5 - Perl6
On Wed, Dec 07, 2005 at 16:48:11 -0500, Peter Schwenn wrote: Dear Perl6 Language, I am Perl user from near year 0. For me the easiest way to learn (, track, and get to the point of contributing to) Perl6 would be a Perl grammar (a regex rule set in, preferably, Perl6) that transforms any Perl5 script into a Perl6. Of couse, besides learning Perl6 for a regex'r or Perl5'r such as myself, and tracking, and documenting 6, it would have huge use for Perl5 users making or considering the transition. IMHO machine translation is not that good a way to start learning - the real benefit of Perl 6 is in the features which have no perl 5 equivalents and solve problems much more elegantly. The best thing to do is to hang out on #perl6 and get involved with the test suite, as well as reading the synopses. Perhaps writing a toy program or something like that could also help. For example one can infer the structure and some of the rules from http://svn.openfoundry.org/pugs/docs/other/porting_howto which is however written out in informal (not regex) rules such as $array[idx] - @array[idx] These are rules for humans with lots and lots of context info... Furthermore, they are more of a perl 6 gotcha list, for perl 5 programmers than a translation guide. Is there such a Perl5-Perl6 translator underway? Larry Wall is working on using the perl (5) interpreter to create compiled output (as opposed to just something that executes in memory) that can then be read by a translator without actually parsing perl 5. Before this happens this will be very very hard - the high level language has vast amounts of implications on execution etc, but the opcode tree is much more simpler to predict (for a computer). p.s. The developing form of such a grammar could likely lead to a grammar package which facilitates rule sets for languages in other domains, in terms of illuminating means of choosing among modes for rule ordering, collecting, scoping, re-application, recursion, exclusion and so forth. Since perl 5's actual parser and tokenizer will be used for this it won't be very extensible, but this is important because perl is reallly hard to parse. -- () Yuval Kogman [EMAIL PROTECTED] 0xEBD27418 perl hacker /\ kung foo master: /me supports the ASCII Ribbon Campaign: neeyah!!! pgpnA5re5zZw2.pgp Description: PGP signature
Re: Perl grammar for Perl5 - Perl6
Peter Schwenn schreef: Is there such a Perl5-Perl6 translator underway? http://dev.perl.org/perl6/ Perl6 FAQ -- Grtz, Ruud
Re: Perl grammar for Perl5 - Perl6
Is there such a Perl5-Perl6 translator underway? I started toying with one, but didn't get very far yet. PPI (on CPAN) would be a good way to start, but it needs to be extended to be used as a Perl5-Perl 6 translator. Greetings, Christian -- [EMAIL PROTECTED] - http://christian.web42.com - http://www.web42.com/crenz/ If the Creator really did write himself into his own story, that's what we ought to expect to see. Creative solutions. And this creativity is intended to be transitive. We are expected to be creative. And we're expected to help others be creative. -- Larry Wall
Re: Perl grammar for Perl5 - Perl6
On Thu, Dec 08, 2005 at 11:51:52AM +0200, Yuval Kogman wrote: : On Wed, Dec 07, 2005 at 16:48:11 -0500, Peter Schwenn wrote: : Dear Perl6 Language, : : I am Perl user from near year 0. For me the easiest way to learn (, : track, and get to the point of contributing to) Perl6 would be a Perl : grammar (a regex rule set in, preferably, Perl6) that transforms any : Perl5 script into a Perl6. Of couse, besides learning Perl6 for a : regex'r or Perl5'r such as myself, and tracking, and documenting 6, it : would have huge use for Perl5 users making or considering the : transition. : : IMHO machine translation is not that good a way to start learning - : the real benefit of Perl 6 is in the features which have no perl 5 : equivalents and solve problems much more elegantly. Except it would be lovely to have a smart enough refactoring translator that it could recognize where those elegant solutions are possible and at least give the option of attempting them. Or at least a hint that there might be a better way. : The best thing to do is to hang out on #perl6 and get involved with : the test suite, as well as reading the synopses. : : Perhaps writing a toy program or something like that could also : help. Sure, but some of our toys are bigger than others. :-) : Is there such a Perl5-Perl6 translator underway? : : Larry Wall is working on using the perl (5) interpreter to create : compiled output (as opposed to just something that executes in : memory) that can then be read by a translator without actually : parsing perl 5. Yes, I have a version of 5.9.2 that dumps out some *very* strange XML that represents, as closely as possible, the exact meaning of the code to Perl 5, along with all the syntactic bits. I then filter that strange XML back into something approximating an AST. I am in the process of proving to myself that I'm getting enough information out of this to recreate the original Perl 5, so I jokingly call this my Perl5-to-Perl5 translator. As of today, I'm able to translate 76.57% of the t/*/*.t files that come with the Perl distribution. Considering that last week this number was down at about 5%, it would seem that I've been making a lot of progress. But most of the work went into that first 5%, and a lot of work will likely go into the last 5% as well. To get that first 5% I basically had to completely refactor the lexer and the grammar without changing anything, which is of course impossible. The Perl 5 parser forgets or misplaces an astounding variety of information that the translator needs, and you can't just go and tell it to turn off the optimizations, because in fact most of those optimizations are deeply interwingled with semantic analysis and transformations as well. Basically, every skipspace() in toke.c and every op_free() in op.c and every rule reduction in perly.y loses necessary information. To attempt to do what I'm currently doing you would have to be completely insane like me. It's a total nightmare. If I were Catholic I'd be hoping this all counts as pennance for my past sins, and gets me out of 100 million years of Purgatory or so. But being a Protestant, I'm merely repenting of my past sins, and thinking about maybe repenting my future ones. And if I were Jewish I'd've said Oy vey many times over. :-) Anyway, once I get to 100% of the t/ files, I'll make it translate all of CPAN back to itself. And at some point I'll take a first whack at the Perl5-to-Perl6 translator, then open it up for community participation. It's still just a bit too early for that, though, because there's such a delicate interplay between refactoring bits of perl without changing anything vs trying to guess whether we are getting enough type and structural information out to recreate the original in the backend. There are already more than 1 lines of code in the backend just to undo the damage done by the Perl 5 engine. : Before this happens this will be very very hard - the high level : language has vast amounts of implications on execution etc, but the : opcode tree is much more simpler to predict (for a computer). Right. But my intent is to write a really good translator, and that implies that it has to be a multi-level translation. That involves keeping track of all the subtle semantic and pragmatic information as well as the basic syntactic information. Otherwise we might as well just feed Perl 5 to babblefish and see if Perl 6 comes out... : p.s. The developing form of such a grammar could likely lead to : a grammar package which facilitates rule sets for languages in : other domains, in terms of illuminating means of choosing among modes : for rule ordering, collecting, scoping, re-application, recursion, exclusion and so forth. : : Since perl 5's actual parser and tokenizer will be used for this it : won't be very extensible, but this is important because perl is : reallly hard to parse. And it's oh about 20 times harder
Re: Perl grammar for Perl5 - Perl6
LW == Larry Wall [EMAIL PROTECTED] writes: LW And if I were Jewish I'd've said Oy vey many times over. :-) or if you were a lazy jewish perl6 hacker you would code: say Oy Vey! for 1 .. ; :) uri -- Uri Guttman -- [EMAIL PROTECTED] http://www.stemsystems.com --Perl Consulting, Stem Development, Systems Architecture, Design and Coding- Search or Offer Perl Jobs http://jobs.perl.org
Re: Perl grammar for Perl5 - Perl6
As for the original question, I think that the Perl 6 grammar will be a much better example for how to parse other languages than a Perl 5 grammar would be, since one of the underlying design currents from the beginning has been that Perl 6 had to be a language that was amenable to parsing by Perl 6 rules (with a little help from a bottom-up operator-precedence expression parser.) Once Patrick is done with PGE, will it be able to parse Perl5? If so, why aren't we focusing on that? Rob
Re: Perl grammar for Perl5 - Perl6
On Thu, Dec 08, 2005 at 01:46:51PM -0500, Rob Kinyon wrote: As for the original question, I think that the Perl 6 grammar will be a much better example for how to parse other languages than a Perl 5 grammar would be, since one of the underlying design currents from the beginning has been that Perl 6 had to be a language that was amenable to parsing by Perl 6 rules (with a little help from a bottom-up operator-precedence expression parser.) Once Patrick is done with PGE, will it be able to parse Perl5? If so, why aren't we focusing on that? PGE isn't much of a parser; it's more like a framework for bulding parsers. So, once PGE is done, to parse perl5, you'll still have to write a grammar-ish thing that includes all of the maze of twisty executions that perl currently goes through to parse perl. The same holds for parsing perl6 only the twisty maze isn't there (or is at least much much smaller) -Scott -- Jonathan Scott Duff [EMAIL PROTECTED]
Re: Perl grammar for Perl5 - Perl6
On Thu, Dec 08, 2005 at 01:46:51PM -0500, Rob Kinyon wrote: : As for the original question, I think that the Perl 6 grammar will : be a much better example for how to parse other languages than a : Perl 5 grammar would be, since one of the underlying design currents : from the beginning has been that Perl 6 had to be a language that : was amenable to parsing by Perl 6 rules (with a little help from a : bottom-up operator-precedence expression parser.) : : Once Patrick is done with PGE, will it be able to parse Perl5? If so, : why aren't we focusing on that? Sure, but if you want to parse it *correctly* and know what it actually means, I estimate it'll take about 20 times as much effort as you're thinking it will, because there are an incredible number of foibles in the Perl 5 parser. Otherwise you might as well just use perl5:PPI. I'm not trying to malign the PPI approach here. It works most of the time, and when it works, it works fine. I don't doubt that it will produce better translations of certain programs, especially source-filtered programs where the filtering can essentially be ignored. But with the PPD approach I'm aiming first for a correct semantic translation and secondarily for a close syntactic translation, so what you'll get out of my approach is a post-source-filtered translation that is likelier to work but less likely to be readable. For a known source filter we can perhaps then refactor that back to a more readable form. In the long run these different approaches will complement each other, and help us better understand each approach's weaknesses and strengths. It's just that I have to take the PPD approach because no one else is simultaneously knowledgable enough and nutty enough to attempt it and succeed. (Whether I am myself knowledgable enough and nutty enough to succeed is yet to be decided. Recent signs are encouraging, especially in the latter department.) Larry
Re: Perl grammar for Perl5 - Perl6
On Thu, Dec 08, 2005 at 01:10:14PM -0600, Jonathan Scott Duff wrote: : The same holds for parsing perl6 only the twisty maze isn't there (or is : at least much much smaller) Hmm. Maybe... Perl 5 is a twistly maze of passages all different. Perl 6 is a twistly maze of passages all alike. Larry
Re: Perl grammar for Perl5 - Perl6
Hi Peter, There are several source filters that make select changes and modules that give select Perl 6 features to Perl 5. All in all, there's a lot there -- I've written Perl6::Contexts, for example, which diddles the bytecode to give Perl 5 programs most of the new contexts from Perl 6. In fact, I kind of even wrote a book on doing this stuff... http://www.perl6now.com. If you download the code listings, you'll get examples of a couple dozen of these modules. As far as your example of $array[idx], see Perl6::Variables. I have non-release version on Perl6::Variables in my CPAN area, SWALTERS, that adds more of the Perl 6 changes. I know this isn't the approach you had in mind, but I thought I'd mention it, as there is actually stuff there to play with. Fundamentally, regexen to transform a Perl program aren't a bad idea, but this approach, of fixing up Perl 5, is less likely to introduce bugs into the code and keeps the production-ready environment that Perl 5 is. Transforming the code isn't a bad idea where there are transformations, but in many cases, there aren't. I suppose that could be a fatal error, but people would still be trying to convert their Perl 5 programs, much as when people where converting to C from B, and any failure scenario would create a flood of help requests, bug reports, etc. I hope this helps! Best regards, -scott On 0, Peter Schwenn [EMAIL PROTECTED] wrote: Dear Perl6 Language, I am Perl user from near year 0. For me the easiest way to learn (, track, and get to the point of contributing to) Perl6 would be a Perl grammar (a regex rule set in, preferably, Perl6) that transforms any Perl5 script into a Perl6. Of couse, besides learning Perl6 for a regex'r or Perl5'r such as myself, and tracking, and documenting 6, it would have huge use for Perl5 users making or considering the transition. For example one can infer the structure and some of the rules from http://svn.openfoundry.org/pugs/docs/other/porting_howto which is however written out in informal (not regex) rules such as $array[idx] - @array[idx] Is there such a Perl5-Perl6 translator underway? Sincerely, Peter Schwenn www.schwenn.com p.s. I'm not yet up to originating a good one, but believe I could make contributions in extending and checking it. p.s. A Perl5 starting point might be better since running it on itself would provide and initiate at least 2 major test modes. p.s. The developing form of such a grammar could likely lead to a grammar package which facilitates rule sets for languages in other domains, in terms of illuminating means of choosing among modes for rule ordering, collecting, scoping, re-application, recursion, exclusion and so forth.
Re: [Phoenix-pm] Perl grammar for Perl5 - Perl6
Oh, sorry... Larry Wall didn't actually to Phoenix-pm. At least not that I know of. This was a forward from me. Thought ya'll might find this interesting... on a Perl 5 level, in a Perl 6 sort of way, and also on the subject of software refactoring. -scott On 0, Larry Wall [EMAIL PROTECTED] wrote: On Thu, Dec 08, 2005 at 11:51:52AM +0200, Yuval Kogman wrote: : On Wed, Dec 07, 2005 at 16:48:11 -0500, Peter Schwenn wrote: : Dear Perl6 Language, : : I am Perl user from near year 0. For me the easiest way to learn (, : track, and get to the point of contributing to) Perl6 would be a Perl : grammar (a regex rule set in, preferably, Perl6) that transforms any : Perl5 script into a Perl6. Of couse, besides learning Perl6 for a : regex'r or Perl5'r such as myself, and tracking, and documenting 6, it : would have huge use for Perl5 users making or considering the : transition. : : IMHO machine translation is not that good a way to start learning - : the real benefit of Perl 6 is in the features which have no perl 5 : equivalents and solve problems much more elegantly. Except it would be lovely to have a smart enough refactoring translator that it could recognize where those elegant solutions are possible and at least give the option of attempting them. Or at least a hint that there might be a better way. : The best thing to do is to hang out on #perl6 and get involved with : the test suite, as well as reading the synopses. : : Perhaps writing a toy program or something like that could also : help. Sure, but some of our toys are bigger than others. :-) : Is there such a Perl5-Perl6 translator underway? : : Larry Wall is working on using the perl (5) interpreter to create : compiled output (as opposed to just something that executes in : memory) that can then be read by a translator without actually : parsing perl 5. Yes, I have a version of 5.9.2 that dumps out some *very* strange XML that represents, as closely as possible, the exact meaning of the code to Perl 5, along with all the syntactic bits. I then filter that strange XML back into something approximating an AST. I am in the process of proving to myself that I'm getting enough information out of this to recreate the original Perl 5, so I jokingly call this my Perl5-to-Perl5 translator. As of today, I'm able to translate 76.57% of the t/*/*.t files that come with the Perl distribution. Considering that last week this number was down at about 5%, it would seem that I've been making a lot of progress. But most of the work went into that first 5%, and a lot of work will likely go into the last 5% as well. To get that first 5% I basically had to completely refactor the lexer and the grammar without changing anything, which is of course impossible. The Perl 5 parser forgets or misplaces an astounding variety of information that the translator needs, and you can't just go and tell it to turn off the optimizations, because in fact most of those optimizations are deeply interwingled with semantic analysis and transformations as well. Basically, every skipspace() in toke.c and every op_free() in op.c and every rule reduction in perly.y loses necessary information. To attempt to do what I'm currently doing you would have to be completely insane like me. It's a total nightmare. If I were Catholic I'd be hoping this all counts as pennance for my past sins, and gets me out of 100 million years of Purgatory or so. But being a Protestant, I'm merely repenting of my past sins, and thinking about maybe repenting my future ones. And if I were Jewish I'd've said Oy vey many times over. :-) Anyway, once I get to 100% of the t/ files, I'll make it translate all of CPAN back to itself. And at some point I'll take a first whack at the Perl5-to-Perl6 translator, then open it up for community participation. It's still just a bit too early for that, though, because there's such a delicate interplay between refactoring bits of perl without changing anything vs trying to guess whether we are getting enough type and structural information out to recreate the original in the backend. There are already more than 1 lines of code in the backend just to undo the damage done by the Perl 5 engine. : Before this happens this will be very very hard - the high level : language has vast amounts of implications on execution etc, but the : opcode tree is much more simpler to predict (for a computer). Right. But my intent is to write a really good translator, and that implies that it has to be a multi-level translation. That involves keeping track of all the subtle semantic and pragmatic information as well as the basic syntactic information. Otherwise we might as well just feed Perl 5 to babblefish and see if Perl 6 comes out... : p.s. The developing form of such a grammar could likely lead to : a grammar package which facilitates rule sets for languages in
