Re: S13: Deep operators

2004-11-25 Thread Michele Dondi
OT
On Wed, 24 Nov 2004, David Ross wrote:
I have been studying PERL 5 core and modules to identify options and
issues for meta-architectures and automated code generation. PERL 6
documents and discussion provide insight essential to effectively using
PERL 5 and preparing for PERL 6.
[snip]
developing in PERL struggling to catch up. The conceptual and concrete
Funny to notice how you could study PERL{5,6} so much still failing to 
realize that there's not such a thing as PERL. See

  perldoc -q 'difference between perl and Perl'
OT
No offense intended,
Michele
--
Your right, I didn't think of that at all, but still, who's gonna go into
the temp internet folder and create a cookie? At least not most users.
Of course *most* users aren't going to do that.  *Most* users aren't
trying to hack your site!  You don't program securely for *most* users -
you program securely for the few users who *are* trying to be malevolent.
- Paul Lalli in clpmisc, Re: free source authentication script


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Michele Dondi
On Thu, 25 Nov 2004, Adam Kennedy wrote:
I thought it was about time I brought some concerns I've been having lately 
to the list. Not so much on any particular problem with perl6, but on 
problems with perl5 we would seem to have the opportunity to fix but aren't. 
(So far as I can tell).
So why not discussing this somewhere else? (e.g. clpmisc)
One of the biggest problems I have had with perl5 is that nothing, not even 
perl itself, can truly actually parse Perl source. By this, I mean parse
False:
[Nothing but] perl can parse Perl. (Tom Christiansen)
Michele
--
# This prints: Just another Perl hacker,
seek DATA,15,0 and  print   q... DATA;
__END__


Re: can gather/take have multiple bins

2004-11-25 Thread Michele Dondi
On Wed, 24 Nov 2004, Dave Whipp wrote:
Juerd point out (private email) that my example doesn't really make any
sense in that it doesn't do anything over and above s/take/push.
However, I think the concept of multiple bins could still be useful. My
What about an adverb? Hope not to say anything utterly wrong/stupid (I've 
not really caught up on Perl6 syntax/semantics yet!), something along the 
lines of

  take $this, $that :in(bin1);
Michele
--
try sleeping on it, that usually works.
I think you're right. Usually it works every time. ;-)
I don't know about that.
I tried sleeping on a big big problem and we're now divorsed.
- Tralfaz on sci.math, Re: About a big big problem (edited)


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Adam Kennedy
Let's say you want to write a yacc grammar to parse Perl 6, or
Parse::RecDescent, or whatever you're going to use.  Yes, that will be
hard in Perl 6.  Certainly harder than it was in Perl 5.
In the end, I concluded there was _no_ way to write even a Perl 5 parser 
using any sort of pre-rolled grammar system, as the language does not 
have that sort of structure.

PPI was done the hard way. Manually stepping through line by line and 
using a variety of cruft (some stolen from the perl source, some my own) 
to make it just work.

I would envisage that the same would be true of writing a PPI6, except 
with a hell of a lot more operators :)

However, Perl 6 comes packaged with its own grammar, in Perl's own rule
format.  So now the quote only perl can parse Perl may become only
Perl can parse Perl  (And even only Perl can parse perl, since it's
written in itself :-).
Perl's contextual sensitivity is part of the language.  So the best you
can do is to track everything like you mentioned.  It's going to be
impossible to parse Perl without having perl around to do it for you.

But using the built-in grammar, you can read in a program, macros and
all, and get an annotated source tree back, that you could rebuild the
source out of.
Again, this is of very little use, effectively destroying the source 
code and replacing it with different source that is a serialised version 
of the tree.

For a current notional example, it would be like loading a simple...
try {
  $object-$do_something;
} catch (Exception $problem) {
  handle($problem);
}
... changing -$do_something to -$do_something() to make it 
back-portable, and then ending up with...

Module::Exceptions::initialize('line 98');
my $exceptionhandler = Module::Exceptions::prepare();
eval {
  $exceptionhandler-update_status('in try');
  $object-do_something();
};
if ( $@ ) {
  if ( ref $exceptionhandler ) {
require Scalar::Util ();
if ( Scalar::Util::blessed $exceptionhandler eq 'Exception' ) {
  do {
my $problem = $exceptionhandler-fetch_exception_as('$problem');
# handler starts here
handler($problem);
$problem-clean_up;
  };
}
  } else {
# Just die as normal
die $@;
  }
}
While technically they may be identical once they get through the parser 
and into tree form, trying to changing -$do_something to 
-$do_something() and getting back some huge monster chunk of code you 
didn't expect is definitely not what the intent of parsing it in the 
first place was.

This is what I am talking about when I refer to the Frontpage effect, 
the habit Micrsoft's HTML editor (especially the early versions) had of 
reuilding you HTML document from scratch, deleting all your template 
variables and PHP code and generally making it impossible to write HTML 
by hand. For HTML where you arn't MEANT to be writing stuff by hand 
under normal circumstances that wasn't always a problem, but perl _isi_ 
meant to be written by hand.

 You could even grab the comments and do something sick
with them (see Damian :-).  Or better yet, do something that PPI
doesn't, and add some sub call around all statements, or determine the
meaning of brackets in a particular context.
The question of whether to execute BEGIN blocks is a tricky one.
Sometimes they change the parse of the program. Sometimes they do other
stuff.  All you can hope for is that people understand the difference
between BEGIN (change parsing) and INIT (do before the program starts).
Frankly that is a gaping security hole... not only do I have to still 
deal with the problem of loading every single dependency or having no 
parsing ability otherwise, but I am required to trust every perl 
programmer on the planet :(

I love PPI, by the way :-)
Thank you, I do to :)
But I'd like to still have something like it in perl6 :(
Adam


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Adam Kennedy
Michele Dondi wrote:
On Thu, 25 Nov 2004, Adam Kennedy wrote:
I thought it was about time I brought some concerns I've been having 
lately to the list. Not so much on any particular problem with perl6, 
but on problems with perl5 we would seem to have the opportunity to 
fix but aren't. (So far as I can tell).

So why not discussing this somewhere else? (e.g. clpmisc)
One of the biggest problems I have had with perl5 is that nothing, not 
even perl itself, can truly actually parse Perl source. By this, I 
mean parse

False:
[Nothing but] perl can parse Perl. (Tom Christiansen)
Please see Acme::BadExample. perl itself cannot parse this at all, and 
yet it follows the absolutely most basic syntax for the language.

And just after the snip you will see I qualify parse in this context 
as loading the perl in some form of DOM-type tree.

Adam


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Adam Kennedy
Smylers wrote:
Adam Kennedy writes:

perl itself would also appear unable to understand perl source,
instead doing what I would call RIBRIB parsing, Read a bit, run a
bit.

RIBRIB?  RABRAB, surely!
Smylers
Yes, you are right, typo.


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Herbert Snorrason
On Thu, 25 Nov 2004 22:00:03 +1100, Adam Kennedy [EMAIL PROTECTED] wrote:
 And just after the snip you will see I qualify parse in this context
 as loading the perl in some form of DOM-type tree.
And yet you disqualify the Perl6 rule system, with its tree of match
objects? What, exactly, is it that you want?

-- 
Schwäche zeigen heißt verlieren;
härte heißt regieren.
  - Glas und Tränen, Megaherz


angle quotes for here-docs ?

2004-11-25 Thread Thomas Seiler
Hi
Is $heredoc = «END; the same as $heredoc = END; ?
thomas


Re: angle quotes for here-docs ?

2004-11-25 Thread Juerd
Thomas Seiler skribis 2004-11-25 14:52 (+0100):
 Is $heredoc = «END; the same as $heredoc = END; ?

I certainly hope not.

Quoting the delimiter is needed, by the way.

How is 'END' disambiguated from 'qw' list, anyway?


Regards,

Juerd


Re: can gather/take have multiple bins

2004-11-25 Thread Rod Adams
Dave Whipp wrote:
gather @bin1, @bin2 - $bin1, $bin2{
 for @words {
   $bin1.take if /^^a/;
   $bin2.take if /e$$/;
 }
}
   

Juerd point out (private email) that my example doesn't really make any
sense in that it doesn't do anything over and above s/take/push.
However, I think the concept of multiple bins could still be useful. My
understand of gather/take is that the function that does the gather is
coroutined (if that's a verb) against the block that does the gathering. So
to be able to usefully use multiple streams, the gather would need to fork
into multiple threads that are then interleaved with the gather block.
Calling a function in junctive context does the forking bit, so a gather
that gathered in a junctive context might make sense as a serialization
mechanism. I can't quite work out what the syntax would be for that though.
Okay, thinking a bit about this, I agree that there are times when you 
want multiple lazy lists going on at the same time. But, frankly if the 
lists are lazy, why shouldn't the programmer be lazy as well?

@bin1 = gather {
 for @words {
take if /^^a/;
 }
}
@bin2 = gather {
 for @words {
take if /e$$/;
 }
}
Two lazy lists? Two gathers!
Since they are lazy, you can eval them in order you please.
I see a problem with attempting to put the two into one statement. In 
your example above, if you wanted something from @bin2, you could have 
to process a whole lot of things into @bin1, just to get the next 
element out of @bin2. In fact, you could easily end up processing all of 
@words, and filling @bin1, only to discover that @bin2 is empty. To me 
this kind of forced-fill of @bin1 is a complete defeat of the entire 
concept of _lazy_ lists.

There's also the issue of from what I understood of gather/take before 
(still awaiting a proper definition), gather returns a lazy list. So 
it's not Cgather @bin1 { ... }, it's C@bin1 = gather {...}. What 
would your Cgather return? A list of lazy lists? How do you easily 
tell the difference between a list of lazy lists and a lazy list? I 
think it's better to leave a 1::1 gather::list ratio, as that will cover 
some {insert randomly high %age} of all cases. For the other cases, 
you're likely better off building a class with two emitter methods, and 
possibly even use Ctie to make it look like two lazy lists.

-- Rod Adams

The thing about lists is that they are linear. They are, by definition, 
a sequence of single items. Now, that 'single item' may very well be a 
reference to something non-trivial, but to the list it's a single value.




Re: angle quotes for here-docs ?

2004-11-25 Thread Larry Wall
On Thu, Nov 25, 2004 at 05:03:36PM +0100, Juerd wrote:
: Thomas Seiler skribis 2004-11-25 14:52 (+0100):
:  Is $heredoc = «END; the same as $heredoc = END; ?
: 
: I certainly hope not.
: 
: Quoting the delimiter is needed, by the way.
: 
: How is 'END' disambiguated from 'qw' list, anyway?

To get the qw// parse you must put a space between the  and the
quote.  This is no hardship semantically, since qw// has always thrown
away initial and trailing whitespace.

Under one way of looking at it, it's just kind of a longest token
rule, since  as a term only ever means qw, and the heredoc tokens
are actually just long opening quotes: ', , q/, etc. that have
to match a trailing quote.

Larry


Hyper Here-Docs? (was: Re: angle quotes for here-docs ?)

2004-11-25 Thread Rod Adams
Juerd wrote:
Thomas Seiler skribis 2004-11-25 14:52 (+0100):
 

Is $heredoc = «END; the same as $heredoc = END; ?
   

I certainly hope not.
Quoting the delimiter is needed, by the way.
How is 'END' disambiguated from 'qw' list, anyway?
 

Seeing the « in the context of a here-doc made me think can you do a 
» here-doc?

So, something like :
@text = »END;
text1
END
text2
END
text3
END
text4
END
for @text { ...}

The hard question about this is: how do you know when you've hit the 
last END? especially if the text you're loading looks like Perl code, or 
if you have different END later in your code?

btw, should it be », «, or »«?
-- Rod Adams


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Larry Wall
On Thu, Nov 25, 2004 at 02:31:46PM +1100, Adam Kennedy wrote:
: Let's say you want to write a yacc grammar to parse Perl 6, or
: Parse::RecDescent, or whatever you're going to use.  Yes, that will be
: hard in Perl 6.  Certainly harder than it was in Perl 5.
: 
: In the end, I concluded there was _no_ way to write even a Perl 5 parser 
: using any sort of pre-rolled grammar system, as the language does not 
: have that sort of structure.

On that level you have to think of Perl as multiple languages, not a
single language.  That in itself should not be a problem, though.

: PPI was done the hard way. Manually stepping through line by line and 
: using a variety of cruft (some stolen from the perl source, some my own) 
: to make it just work.
: 
: I would envisage that the same would be true of writing a PPI6, except 
: with a hell of a lot more operators :)

The number of operators is a bit of a red herring.  What you really
don't like is that there aren't a fixed number of them.  :-)

: However, Perl 6 comes packaged with its own grammar, in Perl's own rule
: format.  So now the quote only perl can parse Perl may become only
: Perl can parse Perl  (And even only Perl can parse perl, since it's
: written in itself :-).
: 
: Perl's contextual sensitivity is part of the language.  So the best you
: can do is to track everything like you mentioned.  It's going to be
: impossible to parse Perl without having perl around to do it for you.
: 
: But using the built-in grammar, you can read in a program, macros and
: all, and get an annotated source tree back, that you could rebuild the
: source out of.
: 
: Again, this is of very little use, effectively destroying the source 
: code and replacing it with different source that is a serialised version 
: of the tree.

And there you put your finger onto the real problem, which is not that
Perl is a mutating language or that it has a lot of operators, but that
in the process of getting from here to there, it *forgets* how it got
there, so there's no way of getting back to here.

: This is what I am talking about when I refer to the Frontpage effect, 
: the habit Micrsoft's HTML editor (especially the early versions) had of 
: reuilding you HTML document from scratch, deleting all your template 
: variables and PHP code and generally making it impossible to write HTML 
: by hand. For HTML where you arn't MEANT to be writing stuff by hand 
: under normal circumstances that wasn't always a problem, but perl _isi_ 
: meant to be written by hand.

But under another view, explosions of opcodes are just part of the
compilation process.  Again, the real problem is the forgetting of
both the original structure and what it means in the context of the
language that was being parsed at the time.

There is no doubt that source filters are much too crude, and forget
way too much.  That's why we're trying to kill them dead in Perl 6.
I think the real question is how far we can push Perl 6's macro system
without forgetting anything you want to know about the structure of
the program.  Obviously AST macros will have an easier time of it
than textual macros.  An AST macro can just automatically attach the
original parse and context as properties on the top of the new AST.

To keep this info around for textual macros will require a bit more
trickery, but we have to do it anyway for activities like debugging.
So if we can see that in the larger context of preserving the entire
compilation audit trail, all the better.

:  You could even grab the comments and do something sick
: with them (see Damian :-).  Or better yet, do something that PPI
: doesn't, and add some sub call around all statements, or determine the
: meaning of brackets in a particular context.
: 
: The question of whether to execute BEGIN blocks is a tricky one.
: Sometimes they change the parse of the program. Sometimes they do other
: stuff.  All you can hope for is that people understand the difference
: between BEGIN (change parsing) and INIT (do before the program starts).
: 
: Frankly that is a gaping security hole... not only do I have to still 
: deal with the problem of loading every single dependency or having no 
: parsing ability otherwise, but I am required to trust every perl 
: programmer on the planet :(

Another red herring--we've always had fairly strict accountability on
the language warping dependencies at the use level.  We're improving
that in Perl 6 by requiring a decision on version at use time, and
making that version a part of the metadata.

But it's no accident that one of the places that Perl 5's B::Deparse
has troubles is right at the BEGIN boundaries.  Wherever Deparse has
troubles, you can read that to mean I didn't understand that I should
put something into Perl 5 to remember something important.  The final
metadata for the compiled program has to be able to tell you which
chunks of program were compiled under which language.  That's just
as important as being able to track back to the 

Re: Hyper Here-Docs? (was: Re: angle quotes for here-docs ?)

2004-11-25 Thread Larry Wall
On Thu, Nov 25, 2004 at 11:59:21AM -0600, Rod Adams wrote:
: Seeing the « in the context of a here-doc made me think can you do a 
: » here-doc?

Nope, you can only hyper operators, not terms.

: So, something like :
: 
: @text = »END;
: text1
: END
: 
: text2
: END
: 
: text3
: END
: 
: text4
: END
: 
: for @text { ...}
: 
: 
: 
: The hard question about this is: how do you know when you've hit the 
: last END? especially if the text you're loading looks like Perl code, or 
: if you have different END later in your code?

I think you have a really good place for a split there.  Then there's
no ambiguity about which is the internal separator and which is the
final delimiter.

: btw, should it be », «, or »«?

Er, can I pick D?

Larry


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Adam Kennedy
Herbert Snorrason wrote:
On Thu, 25 Nov 2004 22:00:03 +1100, Adam Kennedy [EMAIL PROTECTED] wrote:
And just after the snip you will see I qualify parse in this context
as loading the perl in some form of DOM-type tree.
And yet you disqualify the Perl6 rule system, with its tree of match
objects? What, exactly, is it that you want?
What I'm after are 3 critical features.
1. You always get back out what you put in.
$source eq serialize(parse($source)).
2. No side effects. Autrijus Tang suggests this may be workable
3. You can parse a document with broken dependencies.
There are a myriad of these situations, such as
- Dependencies you don't have
- Editing on different platform to execution platform (think Win32:: or 
S390/mainframe/GridComputing)

- Unfinished code
- Things you can't get installed (ImageMagick etc)
- Example code that will never be executed
(Imagine if you will a mod_perl syntax highlighting module for 
search.cpan.org. Should the search.cpan.org host have to _install_ every 
single one of the modules in CPAN?)

PPI can do all of these 3 things. Not 100% reliably, but for normal 
code (where normal is actually defined fairly broadly).

In any case, I would like to suspend this debate for a week, as I'll be 
talking with Damian (hopefully) at YAPC.AU. I'll report back afterwards, 
having hopefully imparted the full extent of my problem.

Perl 6 rules or some variation therein may indeed be what I'm after, 
although I need to find out more about the internals.

Do we have a working version yet I can create some demonstrations with?
Adam


Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Damian Conway
Adam Kennedy wrote:
What I'm after are 3 critical features.
1. You always get back out what you put in.
$source eq serialize(parse($source)).
As Larry pointed out, this will depend on how much metadata your parser 
augments your parse-tree with. I think it will be doable (probably by 
subclassing the standard Perl parsing grammar).


2. No side effects. Autrijus Tang suggests this may be workable
No side effects of what? Of parsing? I don't think that's possible. Perl is 
defined such that compile-time side-effects can alter the syntax (and hence 
the parsing) of a program.


3. You can parse a document with broken dependencies.
This is clearly not possible in Perl (5 or 6) in the general case. Perl isn't 
that kind of language.


PPI can do all of these 3 things. Not 100% reliably, but for normal 
code (where normal is actually defined fairly broadly).
And Perl 6's Cgrammar Perl will be able to give you the same.

In any case, I would like to suspend this debate for a week, as I'll be 
talking with Damian (hopefully) at YAPC.AU. I'll report back afterwards, 
having hopefully imparted the full extent of my problem.
I believe I understand your problem pretty well already.
But I'll be more than happy to discuss this whole issue with you.

Perl 6 rules or some variation therein may indeed be what I'm after, 
although I need to find out more about the internals.
So do we. That's why we're building it at the moment. ;-)
See the mailto:[EMAIL PROTECTED] mailing list.

Do we have a working version yet I can create some demonstrations with?
No. We're working on that. There's a partial prototype that runs under 5.8.3 
(but is broken under earlier and later releases) on CPAN as Perl6::Rules.
You should also (re)read Apocalypse 6 and especially Synopsis 6 on 
http://dev.perl.org

Damian


Re: angle quotes for here-docs ?

2004-11-25 Thread Juerd
Larry Wall skribis 2004-11-25  9:39 (-0800):
 : How is 'END' disambiguated from 'qw' list, anyway?
 To get the qw// parse you must put a space between the  and the
 quote.  This is no hardship semantically, since qw// has always thrown
 away initial and trailing whitespace.
 Under one way of looking at it, it's just kind of a longest token
 rule, since  as a term only ever means qw, and the heredoc tokens
 are actually just long opening quotes: ', , q/, etc. that have
 to match a trailing quote.

Is whitespace between q and the delimiters still valid? Would it be
after ? Would that mean using q as the first element in a qw-list will
hurt?

I hope the first answer is 'no' and all other questions are thus
rendered irrelevant. Not that I think getting rid of alphanumeric
delimiters is a good idea, though. I just fear complex parsing and
seemingly random syntax errors.


Juerd


Angle quotes and pointy brackets

2004-11-25 Thread Juerd
As we now know, in many situations,  and « mean the same thing. In
exactly those situations, the same is true for  and ». However,
sometimes, « cannot be used where  can. Here-docs are an example.

«» (or , if you wish) quotes. I am assuming that «» is a shorthand
for qw«», except where special syntax is used with hash slices and
:-pairs, just like //, which is short for m//,  for qq, etcetera.

But as « foo bar » and  foo bar  are the same thing, I wonder what
qw foo bar  means. Is that qw/ foo bar / or is that qw/foo bar/?
And is this consistent with other operators, i.e. rx«» versus rx?

Another question comes to mind as I am typing this message. Can « and 
be used together, or does « always need » and  need ? If a matching
pair is required, then does the same hold true for vector ops with anqle
quotes on both sides (i.e. is that seen as a quoted operator, or as an
operator that happens to have two vectorizing symbols)?

One last question for now: how hard will it be to implement a grammar
with certain not otherwise specified language features *removed*?


Juerd


Re: Angle quotes and pointy brackets

2004-11-25 Thread Smylers
Juerd writes:

 As we now know, in many situations,  and « mean the same thing. In
 exactly those situations, the same is true for  and ». However,
 sometimes, « cannot be used where  can. Here-docs are an example.

Why can't « be used for here-docs?  I thought Larry had said they were
completely interchangeable.

 But as « foo bar » and  foo bar  are the same thing, I wonder what
 qw foo bar  means. Is that qw/ foo bar / or is that qw/foo bar/?

I'd hope it's the former -- that is, that « can be substituted for 
anywhere that  is a single operator, not just somewhere that those two
characters happen to be adjacent to each other in the source, and »
likewise.  Otherwise you could have ridiculous things like:

  mfoo0

which parses as:

  m/foo/  0

being written as:

  mfoo»0

And that's blatantly of no use to anybody.

Smylers



Re: Will _anything_ be able to truly parse and understand perl?

2004-11-25 Thread Luke Palmer
Adam Kennedy writes:
 Herbert Snorrason wrote:
 On Thu, 25 Nov 2004 22:00:03 +1100, Adam Kennedy [EMAIL PROTECTED] wrote:
 
 And just after the snip you will see I qualify parse in this context
 as loading the perl in some form of DOM-type tree.
 
 And yet you disqualify the Perl6 rule system, with its tree of match
 objects? What, exactly, is it that you want?
 
 What I'm after are 3 critical features.
 
 1. You always get back out what you put in.
 $source eq serialize(parse($source)).
 
 2. No side effects. Autrijus Tang suggests this may be workable
 
 3. You can parse a document with broken dependencies.
 There are a myriad of these situations, such as

I'm afraid that's just not possible.  And by that I don't mean very
hard to implement.  I mean impossible, in a halting problem sort of
way.  Take this module:

module OrBar;

macro *infix:{''} (AST $left, AST $right) {
return {
my ($le, $re) = ($left.run, $right.run);
$le * $re - ($le + $re);
}
}

And this program:

use OrBar;

print 3  4;

There's no way you can not have that dependency and still parse the
program.  What if the macro declaration declared it to be lower
precedence that listops?

And as far as not executin BEGIN blocks, well, you can't do that either.
A BEGIN block might execute and do the same thing as that declaration
(all declarations are just shorthands for the appropriate BEGIN blocks).
And then you can't parse without executing it.

You might be able to use whatever Parrot decides the alternative to
Safe.pm is in order to make sure nobody writes:

BEGIN { system 'cat /dev/urandom  /dev/mem' }

But, AFAICT, there's nothing else you can do.  And the language isn't
going to change to solve this problem.  Munging the grammar and the
operators at compile-time is simply too cool.  The single feature I like
most about perl (it's very hard to decide) is its ability to execute
code at compile time... on top of everything you can do at compile time,
of course.

Or you could tell the parser not to execute BEGIN blocks, and hope that
works.

 (Imagine if you will a mod_perl syntax highlighting module for 
 search.cpan.org. Should the search.cpan.org host have to _install_ every 
 single one of the modules in CPAN?)

No, it'll just have to guess, as all syntax highlighters do.  Chances
are, most modules aren't going to change things drastically enough to
make your syntax highlighting all wrong.

But you don't really need to parse to syntax highlight, either.  You
just need to tokenize.  The way the parser will be designed includes an
explicit tokenizer, and don't worry, we'll let you hook into it.  In
fact, it seems PPI is more of a tokenizer than a parser, and that's much
more easily done.  It's also quite easy to recover if it messes up.

Oh, I wrote this after missing your I'd like to suspend this debate for
a week.  So DELETED!

Luke


Re: Angle quotes and pointy brackets

2004-11-25 Thread Juerd
Larry Wall skribis 2004-11-25 13:45 (-0800):
 Hmm, I would say that  is short for qq//, not qq.  Quote characters
 lose their identity when used with generalized quotes.  (I realize this
 is not always true with Perl 5, but that can be construed as a mistake.)
 So «» is not really short for qw«» unless you take the delimiters of the
 latter construct as simple characters without any «» baggage, including
 the need to have a  workaround.  So I'd rather say «» is short for qw//.

I'm happy to read this. Perl 5's semantics with qx|m|qr|s and ''
probably made me translate  to qq instead of qq//, or qq{} as perlop
lists it.

 : But as « foo bar » and  foo bar  are the same thing, I wonder what
 : qw foo bar  means. Is that qw/ foo bar / or is that qw/foo bar/?
 : And is this consistent with other operators, i.e. rx«» versus rx?
 It means qw/ foo bar/, and yes, that's consistent.

That's a relief :)

 This approach doesn't help the person who can't even *display* «», but
 that problem will be solved before the input problem is.  For instance,
 PerlMonks has no problem displaying «», but I haven't a clue how to type
 it into my browser yet.

Should you happen to use X with the Xkb extension, it is a matter of
assigning a key to Multi_key and then typing Multi_key  .

I have assigned my rightmost Windows key (the Menu key) with:

xmodmap -e keysym Menu = Multi_key

 So you want to violate Liskov substitutability on grammars, eh?  :-)

I'd even violate gravity, if I could!

 While one can certainly redefine rule methods to pitch a fit if called,
 the real way you cut down the language is by not referring to those
 rules in the first place from elsewhere.  Which means you have to override
 those referring rules, after which it almost doesn't matter if the
 previously referred to rules are somehow cancelled or not.

I was afraid that that'd be the answer.

 The other part of it is that some of the constructs are catalogued in
 hashes and arrays rather than in rule alternatives.  When you derive
 a grammar you can certainly copy over a part of the hash or array and
 leave out other parts.  These hashes and arrays are loaded up in the
 first place via the various syntactic categories we go on about.  So
 maybe we have some way of cancelling syntax.

That's better news :)

 BEGIN { undef circumfix:« »; }

But if mixed « is allowed, isn't that «»syntax error? Or did I
misinterpret the answer re mixing them?

 my macro circumfix:« » is gone;

Perhaps is gone is a bit too easy for something that shouldn't be
done.


Juerd


Re: Angle quotes and pointy brackets

2004-11-25 Thread Larry Wall
On Thu, Nov 25, 2004 at 11:12:32PM +0100, Juerd wrote:
: But if mixed « is allowed, isn't that «»syntax error? Or did I
: misinterpret the answer re mixing them?

Uh, I wasn't aware that I'd actually answered the question.  :-)

My actual inclination is to disallow it.  I was just trying to argue
myself out of that position.  Bad habit of mine, thinking out loud.

:  my macro circumfix:« » is gone;
: 
: Perhaps is gone is a bit too easy for something that shouldn't be
: done.

Okay, make it is obliterated_after_grave_consideration_of_the_consequences.

Larry


Re: Angle quotes and pointy brackets

2004-11-25 Thread Smylers
Larry Wall writes:

 PerlMonks has no problem displaying «», but I haven't a clue how to
 type it into my browser yet.

If your browser is using Gnome then holding down Ctrl+Shift while typing
AB (for «) or BB (for ») might work.  (This is also working for me
typing this in 'Vim' in a 'Gnome Terminal', but isn't as nice as the
'Vim' digraphs.)

Smylers



Re: angle quotes for here-docs ?

2004-11-25 Thread Larry Wall
On Thu, Nov 25, 2004 at 09:37:21PM +0100, Juerd wrote:
: Larry Wall skribis 2004-11-25  9:39 (-0800):
:  : How is 'END' disambiguated from 'qw' list, anyway?
:  To get the qw// parse you must put a space between the  and the
:  quote.  This is no hardship semantically, since qw// has always thrown
:  away initial and trailing whitespace.
:  Under one way of looking at it, it's just kind of a longest token
:  rule, since  as a term only ever means qw, and the heredoc tokens
:  are actually just long opening quotes: ', , q/, etc. that have
:  to match a trailing quote.
: 
: Is whitespace between q and the delimiters still valid? Would it be
: after ? Would that mean using q as the first element in a qw-list will
: hurt?
: 
: I hope the first answer is 'no' and all other questions are thus
: rendered irrelevant. Not that I think getting rid of alphanumeric
: delimiters is a good idea, though. I just fear complex parsing and
: seemingly random syntax errors.

Whitespace is certainly not allowed after q.  Whether that means a
bare q should disallow whitespace is another matter.  Certainly we've
taken the step of disallowing whitespace between a sub name and its
parentheses, so it's possible we could do it to q// as well, as long as
it's still possible to put whitespace between a two-part quote:

tr[a-z]
  [A-Z];

But allowing the first whitespace was just a matter of being consistent.
I don't think it's a consistency we have to be consistent about, though.

Larry


Re: Angle quotes and pointy brackets

2004-11-25 Thread Alexey Trofimenko
On Thu, 25 Nov 2004 13:45:51 -0800, Larry Wall [EMAIL PROTECTED] wrote:
...
Hmm, I would say that  is short for qq//, not qq.  Quote characters
lose their identity when used with generalized quotes.  (I realize this
is not always true with Perl 5, but that can be construed as a mistake.)
So  is not really short for qw unless you take the delimiters of the
latter construct as simple characters without any  baggage, including
the need to have a  workaround.  So I'd rather say  is short for  
qw//.
...
ah, I forget, how could I do qx'echo $VAR' in Perl6? something like  
qx:noparse 'echo $VAR' ?
(Note: I like thoose adverbs.. I could imagine that in Perl6 if you want  
to have something done in some_other_way, you just should insert  
:some_other_way adverb, and that is! perl will DWIM happily :)

...
This approach doesn't help the person who can't even *display* , but
that problem will be solved before the input problem is.  For instance,
PerlMonks has no problem displaying , but I haven't a clue how to type
it into my browser yet.
...
I notice that in Perl6 thoose funny  and  could be much more common than  
other paired brackets. And some people likes how they look, but nobody  
likes fact that there's no (and won't!) be a consistent way to type them  
in different applications, wether it's hard or easy.

But to swap  with [] or {} could be real shock for major part of people..
We also have another ascii pair,  and  . maybe they could be better than  
 and  ?:) i'm not that farseeing, but isn't problem of distinguishing   
as a bracket and  as an comparison operator no harder than distinguishing  
 as bracket and as part of heredoc?..

or maybe even we could see consistant to go after + + and alike, and  
make old  and  written as + and + (and then lt and gt suddenly could  
become ~ and ~ :)

But I certain, Larry already weighted exact that solution years ago..
P.S. If you have an urgent need to throw spoiled eggs at me, consider all  
above as very late or very early fools day joke.. or you could try, but  
i've never heard about ballistic transcontinental eggs.


$ @ and %

2004-11-25 Thread Alexey Trofimenko
As far as I understood, arrays and hashes, and references them are much  
more similar in Perl6 than it was in Perl5.

F.e. we have @a and $a = [EMAIL PROTECTED];
the same:
 push @a,1,2,3   push $a, 1,2,3
 $b = @a $b = $a
(?)  say @a[]  say $a[]
(?)  myfunc( [EMAIL PROTECTED])myfunc ( *$a)
hm.. i'm not so competent to continue that list.. could anyone kind make a  
comparison table?

in all other places where they work different, we could use @$a in place  
of @a, right?

so I can consider @ as threat as container specificator, and nothing  
more.. (in abstract, of course). And we just happen to have arrays with  
specificator  (@) and name without sigil (a), contrary to scalars which  
always have to had $ in name ($a).

(
  I must admit that I like to have three different namespaces for  
arrays,hashes and scalars. In Perl5. the _only_ thing I miss is ability to
  sub foo (@@) {
   (\my @a, \my @b) = \(@_);
   ...
  }
)

but perl6 has somewhat different data model. And we going to use  
references much more often and easier than in perl5. And we going to have  
much more different container types, not only arrays and hashes. Why to  
threat two containers as first class citizens, and all other as mere  
scalars with references to objects?

so, yeah, it's way too late to make significant changes, but could  
somebody tell me what was wrong with the following models, apart that they  
are different from perl5 (and could make it harder to write perl5 to perl6  
translators)

I'm talking about unifying namespaces of arrays, hashes and scalars. I  
could swear i've seen some RFC about it..

1. all variables are scalars, and as scalars they include $ in their name
2. if scalar $var contains reference to array or hash, we could threat it  
as array or hash using @$var and %$var(or maybe just @var and %var, as  
shorthands).
With all associated behavior, so @var = ... makes list context to right  
side of assignment. We could think of it as of Array operator, or macro.

2. my @a is ... does right thing: it creates lexical $a, places reference  
to anonimous array in it, and apply all the traits as current perl6 would  
do (I'm not sure what differences of traits applying to container and  
scalars, though). So my @a of ..  is just a syntactic sugar to my $a is  
Array of ...

3. subscripts are written as $var{} and $var[], as we have now and always  
had before. still @var{...} could be easily recognized as slice.

as far as I can see, we have one thing to lose:
 1) one namespace instead of three,
(my $x=$x[$i]; It's a Perl! (c))
and several good points:
 1) simplicity.
 2) eliminating of redundancy. We could write all the code involving named  
arrays and hashes using _only_ named scalars with references to arrays and  
hashes, so why to have both ways?.

 3) no logical separation between CORE and extrnal data containers (yeah,  
i know about tying, but why we have separate classes and tyers in perl5?  
is there reason to have write that (tied $var)-method.. in my proposal it  
would be matter of difference between $var.method and @var.method;

 4) using references to containers isn't harder then using containers  
theyself, because it's a same.

 5) one namespace instead of three.
There's plenty of other classifications of variables which would want to  
have separate nonclashing namespaces, (global/private, instance/class  
etc.) but we for whateer reason stick with classifing thousands of  
different container types as three major.

 6) syntax would be much the same as what perl5 programmers used to see.  
just try to look on your perl5 code in my way, and it would work! The only  
thing that they should keep in mind is that $a and @a is the same..

ah, proposed model could prevent some minor optimizations.. but all thoose  
optimizations are gone too if you going to use references to arrays  
instead of arrays itself, aren't they? and if not, than there's no  
trouble..

P.S Yeah, I see, it looks closer to Certain Other Languages. but why not  
if it's simplier and maybe even better? and using @ and % as shorthand to  
that clumsy and verbose to is Array or is Hash is cool, I think, and  
it could be a distinctive feature, which could help us to withstand bitter  
of being less distinctive.

P.P.S I had never feel enough inner strength in myself to be a  
revolutioner. So I suppose I just don't see something important.. so bring  
me to Right Way, please..

P.P.P.S. If answer on my why? would be just because! I would take it  
silently.

P.P.P.P.S
 open $file, filename;
 print @file;
...
P.P.P.P.P.S. oooh.. sh$t! i forgot about @_ ! :) but maybe $_ is an Args  
object? so $_[0] is here, scalar $_ is here for little unprototyped blocks  
in map and others, and even $_{...} syntax for named args too, it's only  
matter of viewpoint?..

P.P.P.P.P.P.S I'm going to write a grammar to write such a perl dialect  
strikeif/strike when you'll decline my proposal.