Re: Ideas that need RFCs?
Dan Sugalski wrote: I expect we'd want to have some sort of heavy-duty regex optimizer, then, to detect common prefixes and subexpressions and suchlike things, otherwise we end up with a rather monstrous alternation sequence... We need a regex merge function too -- then we could write macros that extend the parser. Domain languages! Lexically scoped of course. ;) Obviously we're going to need a new expansion for "regex" because these things aren't regular expressions anymore. (Not that Perl has had regular expressions for a long time...) Really easy grammar experiments? - Ken
Re: Ideas that need RFCs?
At 10:35 AM 8/19/00 +1000, Damian Conway wrote: However, for Perl 6 I'd really like to see run-time access to the Real Tokenizer (tm): use tokenizer; my $tree = tokenizer( $sourcecode ); This would be dead handy for building source-code filtering modules, rather than relying on the pseudo-parsing tricks we're currently forced to use. Well, sure. You couldn't write an optimizer, bytecode compiler, or bytecode execution engine in perl if you couldn't do that... :) Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Ideas that need RFCs?
Larry Wall wrote: Dan Sugalski writes: : At 10:35 AM 8/19/00 +1000, Damian Conway wrote: : However, for Perl 6 I'd really like to see run-time access to the : Real Tokenizer (tm): : : use tokenizer; : : my $tree = tokenizer( $sourcecode ); : : This would be dead handy for building source-code filtering modules, : rather than relying on the pseudo-parsing tricks we're currently : forced to use. : : Well, sure. You couldn't write an optimizer, bytecode compiler, or bytecode : execution engine in perl if you couldn't do that... :) I was wondering this morning whether we ought to write the Perl 6 parser as a set of recursive regexes. Might make it easier to plug in new productions on the fly. And designing the parser around regexes might indicate ways in which Perl's regexes are not yet powerful enough. Larry That would be coolness. -- David Corbin Mach Turtle Technologies, Inc. http://www.machturtle.com [EMAIL PROTECTED]
Re: Ideas that need RFCs?
At 11:18 AM 8/23/00 -0700, Larry Wall wrote: Dan Sugalski writes: : At 10:35 AM 8/19/00 +1000, Damian Conway wrote: : However, for Perl 6 I'd really like to see run-time access to the : Real Tokenizer (tm): : : use tokenizer; : : my $tree = tokenizer( $sourcecode ); : : This would be dead handy for building source-code filtering modules, : rather than relying on the pseudo-parsing tricks we're currently : forced to use. : : Well, sure. You couldn't write an optimizer, bytecode compiler, or bytecode : execution engine in perl if you couldn't do that... :) I was wondering this morning whether we ought to write the Perl 6 parser as a set of recursive regexes. Might make it easier to plug in new productions on the fly. And designing the parser around regexes might indicate ways in which Perl's regexes are not yet powerful enough. Now that's an interesting idea. It might make it easier to extend than the "build a static dispatch table" that seems to be the way yacc-ish things go. It'd also give us a good reason to optimize the heck out of some parts of the regex engine if we want any sort of speed from the thing. I expect we'd want to have some sort of heavy-duty regex optimizer, then, to detect common prefixes and subexpressions and suchlike things, otherwise we end up with a rather monstrous alternation sequence... Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Ideas that need RFCs?
I was wondering this morning whether we ought to write the Perl 6 parser as a set of recursive regexes. That would also solve one of my nagging headaches: RFC XXX: Parse::RecDescent 2.0 *is* Perl 6.0 Of course, then we'd have to have Perl 6 out "by Christmas"! ;-) Damian PS: I case anyone missed it, I was pointing out that beefing up the regex engine until it was capable of implementing fully-fledged parsers would make it vastly more useful to those of us who are fully-fledged parsers. If this were done, I would envisage that the RecDescent module would simply become a kinder, gentler interface to the horrors of the new regex syntax that would be required. The various parsing modules might also be a source of useful ideas and metaphors for the new regex mechanism.
Re: Ideas that need RFCs?
I was wondering this morning whether we ought to write the Perl 6 parser as a set of recursive regexes. Might make it easier to plug in new productions on the fly. And designing the parser around regexes might indicate ways in which Perl's regexes are not yet powerful enough. There's a very nice section on that stuff in "Algorithms in SNOBOL4" (AiS4) on doing just that. The extension that he uses to get around some of the problems in parsing a context-dependent language is defining a "syntactic routine" that gets called at the appropriate point in the pattern match to decide whether or not the given token is of the type needed. This is done by using SNOBOL's dynamic function evaluation and conditional assignment during a pattern match. To do this kind of thing in Perl, we'd need to be able to match a substring, and then call an arbitrary function in the middle of a pattern match, and to back out the call if the match failed. AiS4 gives a lot of funky details on how SNOBOL did this internally, which I can supply if there's any interest... But the thing you get out of this is that you can do all of your code generation during the pattern match. Since you do the match and schedule (but don't execute) the function calls untill the whole match succeeds, you can do arbitrary look-ahead. You can use conditional assignment to handle stacking (conditionally) stuff like the branch over the contents of an if() (if the condition is not true). Is this just of historical interest, or is it useful? I'm not a big-league parser person, just an unregenerate SNOBOL programmer with all these synapses full of old code... ---Joe M.
Re: Ideas that need RFCs?
"Joe" == Joe McMahon [EMAIL PROTECTED] writes: Joe This is done by using SNOBOL's dynamic function evaluation and Joe conditional assignment during a pattern match. To do this kind of Joe thing in Perl, we'd need to be able to match a substring, and Joe then call an arbitrary function in the middle of a pattern match, Joe and to back out the call if the match failed. Already done in 5.6. :) "perldoc perlre". -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 [EMAIL PROTECTED] URL:http://www.stonehenge.com/merlyn/ Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Re: Ideas that need RFCs?
Randal L. Schwartz writes: : "Joe" == Joe McMahon [EMAIL PROTECTED] writes: : : Joe This is done by using SNOBOL's dynamic function evaluation and : Joe conditional assignment during a pattern match. To do this kind of : Joe thing in Perl, we'd need to be able to match a substring, and : Joe then call an arbitrary function in the middle of a pattern match, : Joe and to back out the call if the match failed. : : Already done in 5.6. :) "perldoc perlre". At the moment, there's no way for the call to "fail". Unfortunately. Larry
Re: RFCs (Re: Ideas that need RFCs?)
On Fri, Aug 18, 2000 at 05:22:17PM -0500, David L. Nicol wrote: RFC: Perl6 is Final. There will Be No Perl7 RFC: Everything is Accessible and Mutable RFC: The perl6 reference implementation, no matter how slow it is, will be written in perl5, in some kind of well defined virtual machine. RFC: It's all exception handling. RFC: Implemnentation: Unified containers are trees of storage nodes, and we do our own memory management with them RFC: Garbage collection: We keep reference counting, with a big node pool, occasionally defragment it if we can, to free the top of it. Almost all of these are internals fodder, not language topics. David, could you re-post this list to -internals and take discussion over there, please? K. -- Kirrily Robert -- [EMAIL PROTECTED] -- http://netizen.com.au/ Open Source development, consulting and solutions Level 10, 500 Collins St, Melbourne VIC 3000 Phone: +61 3 9614 0949 Fax: +61 3 9614 0948 Mobile: +61 410 664 994
Re: RFCs (Re: Ideas that need RFCs?)
Nathan Torkington writes: : Steve Fink writes: : We are NOT here to construct a radically better language. We are here to : design the underpinnings of one. : : Perhaps. And by "perhaps", I mean "no". : : We're here to say what we'd like to see in the next version of Perl. : These can be big things (currying) or small (hashes returned by : functions instead of long lists). We're giving input to Larry, who : will then design the language. We are just telling Larry what we : would like, and why (i.e., which itch it would scratch). And my current role is to turn everybody's view upside-down by pointing out that currying is a small thing if you have the right hooks between the lexer and the parser, while returning hashes instead of long lists is a big thing if it implies the unification of object and hash notations. How exactly do you "slice" an object, currently? : If you have an idea that will "add value" to Perl6 but can just as : well be done after the groundwork for the language has been laid : out, then please do not write up an RFC on it. It'll just distract. : : I completely disagree. If you want something in Perl, now's the time : to ask. We're going to have to nail down the language so people can : begin writing grammars, data structures, regex engines, and so on. : There's no such thing as a small change if that change comes *after* : people have begun coding. That's called "feature creep", as I'm sure : you know. I don't mind "small" RFCs. I expect some number of them will be end up being classified as: "Should be implemented in terms of X later." But that doesn't mean they don't encode real desires. : So I want to encourage people to submit RFCs. Yes, there are a lot of : them. That's Larry's problem, not ours. It's one problem he's glad : to have, I'm sure. Number 5 is alive! Not disassemble! Need more input! Larry
Re: RFCs (Re: Ideas that need RFCs?)
Nathan Torkington wrote: Steve Fink writes: We are NOT here to construct a radically better language. We are here to design the underpinnings of one. Perhaps. And by "perhaps", I mean "no". We're here to say what we'd like to see in the next version of Perl. These can be big things (currying) or small (hashes returned by functions instead of long lists). We're giving input to Larry, who will then design the language. We are just telling Larry what we would like, and why (i.e., which itch it would scratch). And both those examples apply to the underpinnings. Ok, maybe I have an unusually broad definition of the word "underpinnings". Think "anything that can't be done with a pure perl module". If you have an idea that will "add value" to Perl6 but can just as well be done after the groundwork for the language has been laid out, then please do not write up an RFC on it. It'll just distract. I completely disagree. If you want something in Perl, now's the time to ask. We're going to have to nail down the language so people can begin writing grammars, data structures, regex engines, and so on. There's no such thing as a small change if that change comes *after* people have begun coding. That's called "feature creep", as I'm sure you know. Which is why I wrote the rest of that paragraph, saying that people should write RFCs for anything that requires, or might require, changes in the Perl6 language. My message was *encouraging* RFCs; I was disagreeing with a message discouraging them. You happened to quote the two sentences where I was agreeing that not all RFCs need to be brought up just now, if the change they propose would still be possible after the Perl6 core was set in stone (namely, you could just write a module to do that using "existing" primitives at no significant loss in performance or clarity. Which usually implies that you could do it today in Perl5, but nobody's bothered to.) So I want to encourage people to submit RFCs. Yes, there are a lot of them. That's Larry's problem, not ours. It's one problem he's glad to have, I'm sure. I agree. At least, to the extent possible without having Larry's brain sitting in front of me in a jar, and I'm pretty sure this one is somebody else's.
Re: RFCs (Re: Ideas that need RFCs?)
Here's my RFC todo list which I am dropping like a good mensch on the occasion of Friday Evening. RFC: Perl6 is Final. There will Be No Perl7 We declare that our framework willbe so flexiblke that anything can be done with it and there will be no penalty for something being in-core opposed to out-of-core and so on. RFC: Everything is Accessible and Mutable From tokenizing up. See below for how to pull this off. At least, the core core core of what's left is just a smidgen larger than FORTH. Like, FORTH plus data structures plus exceptions plus garbage collection. RFC: The perl6 reference implementation, no matter how slow it is, will be written in perl5, in some kind of well defined virtual machine. It should be possible do Data::Dumper out a emulated perl6 instance and load it into another and there you are, except the file handles are all confused (unless you've got a way-fancy OS that can cope with such things) RFC: It's all exception handling. I imagine the core syntax description as a set of catch clauses. Every token generates a "TOKEN-$whatever" exception, which is caught according to the current situation. How's that for a general paradigm? These things can be overloaded as needed to implement Macros, variant syntaces, variant semantics, and so on. RFC: Implemnentation: Unified containers are trees of storage nodes, and we do our own memory management with them RFC: Garbage collection: We keep reference counting, with a big node pool, occasionally defragment it if we can, to free the top of it. Have a nice weekend. Nathan Torkington wrote: Steve Fink writes: We are NOT here to construct a radically better language. We are here to design the underpinnings of one. Perhaps. And by "perhaps", I mean "no". We're here to say what we'd like to see in the next version of Perl. These can be big things (currying) or small (hashes returned by functions instead of long lists). We're giving input to Larry, who will then design the language. We are just telling Larry what we would like, and why (i.e., which itch it would scratch). If you have an idea that will "add value" to Perl6 but can just as well be done after the groundwork for the language has been laid out, then please do not write up an RFC on it. It'll just distract. I completely disagree. If you want something in Perl, now's the time to ask. We're going to have to nail down the language so people can begin writing grammars, data structures, regex engines, and so on. There's no such thing as a small change if that change comes *after* people have begun coding. That's called "feature creep", as I'm sure you know. So I want to encourage people to submit RFCs. Yes, there are a lot of them. That's Larry's problem, not ours. It's one problem he's glad to have, I'm sure. Nat -- David Nicol 816.235.1187 [EMAIL PROTECTED] Does despair.com sell a discordian calendar?
Re: RFCs (Re: Ideas that need RFCs?)
Steve Fink writes: And both those examples apply to the underpinnings. Ok, maybe I have an unusually broad definition of the word "underpinnings". Think "anything that can't be done with a pure perl module". I'm not wild about that metric, either. Exporter is pure Perl, but I'd love to see its functionality moved into the core (or, indeed, replaced by a general compile-time interface setup with separate runtime execution semantics) for speed reasons. My message was *encouraging* RFCs And once again we're in violent agreement :-) Sorry for any confusion, Nat
Re: Ideas that need RFCs?
Stephen P. Potter writes: * The match operator, Cm, is always required (bare C// becomes a fatal error). I could live with that. Damian's done some work trying to tokenize Perl and knows what the weird edge cases are. Damian, can you post your short list? * Replace Cm//, Ctr///, and Cs/// with equivalent regularized functions that take mulitple arguments instead of using specialized syntax. It would be best if the names could be more "complete", like match(), translate(), and substitute() (although translate and substitute are rather long). I think you'll lose that battle. Perl's tight RE integration is a plus, not a minus. There are already long names for matching and substitution if you use the Regexp module. * Disallow use of C/ as delimiters for quote functions and require use of matching pair characters. Not sure what this would win. It's not really a parsing problem. * Remove special meaning from C/. Integrate that functionality with C' and C". Which special meaning? Division? :) This comes under the m-always-mandatory category. Nat
Re: RFCs (Re: Ideas that need RFCs?)
On Fri, 18 Aug 2000, David L. Nicol wrote: There will Be No Perl7 Of course not. Odd numbers are the development releases. The next Perl after 6 will be 8. Seriously, while a worthwhile goal, this is rather short-sighted. The industry and the world will continue to change in spite (or because!) of our efforts here. We can make it easier for the users to adapt, but Perl will need to continue to evolve, as well. (As spoken by a one-eyebrow, knuckle-dragging Neanderthal) -- Bryan C. Warnock ([EMAIL PROTECTED])
Re: Ideas that need RFCs?
: Text::Balanced has to track in order to parse almost any Perl code.
It's a good thing you said "almost".
Ah, that Jesuit education finally pays off! ;-)
Does it parse [*]] correctly?
No, but only because Cextract_variable ignores punctuation vars entirely.
It would be a simple fix to allow it to handle those too.
It does handle C[*{']'}], though. :-)
Does Text::Balanced take into account functions declared with a
prototype of ()?
No. It ignores the function prototypes. Again, that could be fixed
easily enough, were the demand there.
However, for Perl 6 I'd really like to see run-time access to the
Real Tokenizer (tm):
use tokenizer;
my $tree = tokenizer( $sourcecode );
This would be dead handy for building source-code filtering modules,
rather than relying on the pseudo-parsing tricks we're currently
forced to use.
Sigh. Of course it's a forthcoming RFC...
Damian
Re: RFCs (Re: Ideas that need RFCs?)
Jeremy Howard wrote: Steve Fink writes: And both those examples apply to the underpinnings. Ok, maybe I have an unusually broad definition of the word "underpinnings". Think "anything that can't be done with a pure perl module". Say "anything that can't be done *fast*enough* with a pure perl module" and you're closer. Right. What he said. Or maybe add "cleanly enough" too. And "easily enough". Or maybe just "anything that wouldn't be better done as a pure perl module." After all, we're shooting for better than "easy things possible, hard things barely possible."
Re: Ideas that need RFCs?
On Thu, Aug 17, 2000 at 01:07:30PM -0400, Stephen P. Potter wrote: * Replace Cm//, Ctr///, and Cs/// with equivalent regularized functions that take mulitple arguments instead of using specialized syntax. It would be best if the names could be more "complete", like match(), translate(), and substitute() (although translate and substitute are rather long). This one puzzles me. Someone suggested that we remove one of the reasons I use perl rather than awk or python? Some of these are mine, some of them are ideas I remember seeing someone else mention, but I don't see RFCs for. Am I missing them, or do they need to be written up? RFC away! -Scott -- Jonathan Scott Duff [EMAIL PROTECTED]
Re: Ideas that need RFCs?
On Thu, Aug 17, 2000 at 01:07:30PM -0400, Stephen P. Potter wrote:
* Replace Cm//, Ctr///, and Cs/// with equivalent regularized
functions that take mulitple arguments instead of using specialized
syntax. It would be best if the names could be more "complete", like
match(), translate(), and substitute() (although translate and substitute
are rather long).
I was thinking of this too. Well, not quite -- I certainly don't want m,
tr, or s to go away (or /regex/ either.) But the =~ bothers me. How
about disallowing m{...} and using m{expr}/.../?
foreach $line (@foo) {
m{$line}/pattern/;
}
That seems better than $line =~ ... but not by much. How about
disallowing m(...) and using
m($line, /pattern/) ? (similar to split)
Hm. Tough to parse. Where do you look for the opening delimiter? More
similar to split:
m(/pattern/, $line)
I like that one. As long as split,m,tr,s can all handle non-/ delimiters
(as split currently cannot). And as long as I can still say
while() { print "found it!" if /needle/ }
when I don't need to specify the variable.
Re: Ideas that need RFCs?
"Stephen P. Potter" [EMAIL PROTECTED] writes: * Socket functions (such as Caccept, Cbind, etc) should be moved from the core to modules/libraries. * Math functions (such as Cabs, Catan2, Ccos, etc) should be moved from the core to modules/libraries. * IPC functions (such as Cmsgctl, Csemctl, Cshmctl, etc) should be moved from the core to modules/libraries. * "User" functions (Cgetpw*, endpw*, Csetpw*, Cgetlogin) should be moved from the core to modules/libraries. * Group functions (Cgetgr*, Cendgr*, Csetgr*) should be moved from the core to modules/libraries. * Network functions (Cgetnet*, Cgethost*, Cgetproto*, etc) should be moved from the core to modules/libraries. * Process functions (Cgetpgrp, Cgetppid, Cgetpriority, Csetpgrp, Csetpriority) should be moved from the core to modules/libraries. I don't think this has anything to do with the language. Whether a function is built-in, known, loadable, findable, user-defined, or whatever makes no difference from language perspective. See RFC26. From implementation perspective, yes, it makes sense. -- Johan
Re: Ideas that need RFCs?
I certainly don't want m, tr, or s to go away (or /regex/ either.) But the =~ bothers me. I don't think we should change everything. There's a big risk that we get "fix-happy" and change stuff that doesn't really make an improvement. Personally, I like =~ because it has a nice analogy to the math operator that makes something "sort of" equivalent to something else. We're getting deluged with RFC's and emails. We should start thinking "will this RFC or idea *add value* to Perl 6?". If not, and it just makes something work differently, it _might_ not be worth an RFC. But I claim to have *no* authority on this issue. Just my($.02). -Nate
Re: Ideas that need RFCs?
Nathan Wiger wrote: We're getting deluged with RFC's and emails. We should start thinking "will this RFC or idea *add value* to Perl 6?". If not, and it just makes something work differently, it _might_ not be worth an RFC. I disagree completely. For one thing, there's no such thing as Perl6. It doesn't exist. What does exist is Perl5, and the primary motivation here should be to clean up and streamline Perl5. Mainly, removing the special cases, bad syntax, and backwards-compatibility swamps that make it so difficult to continue improving it. We are NOT here to construct a radically better language. We are here to design the underpinnings of one. If you have an idea that will "add value" to Perl6 but can just as well be done after the groundwork for the language has been laid out, then please do not write up an RFC on it. It'll just distract. If you have an idea that will help unify disparate mechanisms, or one that will require syntactic or infrastructural changes, then please DO submit an RFC. Otherwise, Perl6 from its inception will prevent your idea from being realized. It's always too late for backwards compatibility breakage with an existing language, and there's a limited time window before Perl6 exists. Removing =~ and making m// behave like a sub qualifies -- an incompatible change could easily prevent this from happening. Even moreso for \m-abc-, not that I'm in love with that idea. This would even free up =~, though it's so ugly I'm not sure why anyone would want it. Maybe let it be a user-defined equality test, to take inspiration from your comment about approximate equality? Dunno, but killing off two unintuitive punctuation characters seems worthwhile by itself.
Re: Ideas that need RFCs?
Decklin Foster wrote: [replying from here since this is the only way I received it] "Myers, Dirk" wrote: $line/pattern/ ; /pattern/ ($line) ; I don't think these should be changed. Here's how I tend to pronouce things: $x = 'foo'; # "x gets foo" /bar/;# "match on bar" $x =~ /baz/; # "x gets matched on baz" C=~ is a special case of "getting" something, where the something in question is an action instead of an expression. This symmetry should be kept. Bleckth! You want to preserve it because of a coincidental double meaning of the English verb "to get"?! $x is not "getting" anything in that example. By that argument, pretty much every operator should have an equals sign in it. @$x is "x gets dereferenced". foo() is "foo gets called". =~ meaning approximate equality makes some sense to me, but this doesn't.
