Re: Rewrite of S21

[Stefan O';Rear]

On Fri, Nov 23, 2012 at 03:24:00PM +0100, Arne Skjærholt wrote:
> Hi all.
> I have rewritten S21 so that Zavolaj/NativeCall is spec. S21 as of
> right now is probably the most thorough documentation of the current
> state of NativeCall.
> 
> Most of the basic functionality is fairly stable I think, and
> implemented for Rakudo. However, there are many things left:
> - Quite a few decisions remain to be made, many of them because I'm
> not sure what is the best path to take. I have tried to label these
> clearly as hypotheticals.
> - My experience is clearly UNIX-centric. Wider platform coverage will
> probably require revising the spec.
> - The spec text itself can probably be improved on. As it stands, I
> think it's probably a curious hybrid of my academic and more
> conversational prose style.
> 
> I'm happy to discuss comments and suggestions on this.

Hopefully next month when I get around to porting NativeCall I'll be able
to review this.

-Stefan


pgp8c7mcpyZPq.pgp
Description: PGP signature

Re: How to make a new operator.

[Stefan O';Rear]

On Sat, Mar 24, 2012 at 06:16:58PM -0700, Jonathan Lang wrote:
> IMHO: if we're going to take loss of precision into account, we should do so 
> explicitly.  I'm a bit rusty, so forgive me if I misuse the terminology: if a 
> number has an epsilon, the epsilon should be attached to it as a trait so 
> that it can be accessed by the program.  This allows all sorts of things, 
> like "close enough" smart-matching and error propagation.  The main question 
> is if Perl should assign a minimum epsilon to all floats by default, or if 
> this should be an "all's fair if you predeclare" type of thing.  

Speaking as an implementor, I think this is quite unlikely.  Some of my
reasons:

1. Interval arithmetic (this is the correct technical term) is very
   inefficient on some popular architectures.

2. After a long computation which is numerically stable *but which Perl
   cannot _prove_ the stability of*, error bounds will be very large, and
   if smart-matching automatically takes them into account, it will tend
   to result in extremely suprising false positives.

3. Adding hidden state to numbers that is not visible by default will make
   debugging harder.  Not hiding the state will make output much noisier.

Larry is free to override this of course.  Also, interval arithmetic ought
to be possible as a module.

-Stefan


signature.asc
Description: Digital signature

Re: N-dimensional arrays and compiler support

[Stefan O';Rear]

On Thu, Mar 22, 2012 at 11:14:54PM +0100, Daniel Carrera wrote:
> Hey,
>
> I have a few slightly related questions:
>
> 1. The semicolon operator would allow Perl 6 to support N-dimensional
> arrays... How would one iterate over that type of array?
> 
> my num @matrix[ 10 ; 10 ; 10 ];
>
> I ask because a natural extension is to add arithmetic operators and
> you have the beginnings of a Matlab-like array language.

I've started trying to implement S09 things this month, and this is
actually one of the big questions that came up for me.  I raised it
on IRC earlier (which is much more active than p6l, fwiw) and I don't
think we actually arrived at a definite answer.

There are three obvious choices for what 'for @matrix { }' means:

* Iterate over slices that fix the leftmost index

* Iterate over all elements in lexicographic order

* Die

I'm currently thinking of the first, because it allows
(map {$_}, @matrix)[0] to mean the same thing as @matrix[0].  But it's
likely there are other nice considerations.

> 2. Do you think Rakudo is likely to get support for N-dimensional
> arrays in... say... the next year or so?

No comment.  If pmichaud stays alive and well, anything is possible.

> 3. Does anyone here know much about Niecza? Can you compare it with
> Rakudo? I am already familiar with the feature support page
> (http://perl6.org/compilers/features) so I am leaving the question
> intentionally vague. I'd be interested in anything that you think is
> interesting (e.g. speed, development style, progress, whatever).

"For some reason it's much less suprising; I almost never have to guess
what niecza will do with a given piece of code."

Seriously, relative to Rakudo:

Performance: Uses less memory, compiles comparably fast, used to
be much faster at runtime but the gap has narrowed considerably.

Platform: Rakudo/Parrot requires a system with a C89 compiler and a
sufficiently 386-like processor (in particular, it wants a flat address
space and equal size code and data pointers).  Niecza requires a CLR
implementation (developed on Mono, tested on Microsoft .NET).

Rakudo can use C libraries, Niecza can use CLR libraries.

Features: Idiosyncratic variance.  Niecza is still generally ahead on
regex support and a few other odds and ends.  Rakudo currently has the
edge on macros, list behavior, native types, the MOP, and working
libraries.

> Cheers,
> Daniel.

-Stefan



signature.asc
Description: Digital signature

Re: Operator precedence fiddling

[Stefan O';Rear]

On Fri, Dec 23, 2011 at 10:59:58PM +1100, Kris Shannon wrote:
> sub infix: ($a, $b = ($a foo 2 * 3 - $a)) is tighter(&infix:<*>)
> {
>   $a == 0 ?? 1 foo 2 * 3 !! $a + $b
> }
> 
> say 1 foo (2 * 3);
> # 7
> say (1 foo 2) * 3;
> # 9
> 
> say 1 foo 2 * 3;
> # 9
> 
> say infix:(1);
> # Niecza: 7
> 
> say 0 foo 0;
> # Niecza: 9
> 
> The way that Niecza does this is to install the operator with default 
> precedence
> before the signature (which means the default is parsed with * binding
> tighter than
> foo) and then twiddling the precedence before the body (which means the $a == > 0
> case is parsed with foo binding tighter than *)
> 
> Is this right?
> Is it necessary (i.e. does the spec mandate the operator existing
> while parsing the
> signature and/or body)?

I seem to recall that there was some spectest or another which required the
operator to exist in the signature.  The test wasn't using precedence, but
at the time the signature is parsed, Niecza can't know that.

It is most definitely required in the body; normal subs can be recursive, and
recursive operators like ! are a common example.

> I'm considering doing the work to get Rakudo to do the tighter/looser
> traits and can't
> find the bits of the spec which might apply to these (admittedly
> bizarre) edge cases.

Another even more bizarre edge case in niecza is that you can use "also"
to change the precedence of an operator in the middle of the body.

> I suppose for completeness I should have added in some more traits
> which used foo
> in an expression - or is that definitely not ok?

I can't see why it should't be ok.

-Stefan


signature.asc
Description: Digital signature

Re: Encapsulating the contents of container types

[Stefan O';Rear]

On Thu, Aug 18, 2011 at 04:06:47PM +0200, Carl Mäsak wrote:
> I was working on the Little Animal Farm game yesterday, and wanted to
> make it totally safe against tampering from the outside. (See
> .)
> 
> I ended up implementing a custom accessor method and a sub to
> deep-clone hashes, just to make sure that data belonging to readonly
> attributes doesn't get changed.
> 
> This worries me.
> 
> Concrete example (useless output left out):
> 
> $ perl6
> > class A { has @.numbers; method add($n) { @!numbers.push($n) } }
> > my $a = A.new
> > $a.add(1); $a.add(2); $a.add(3)
> > $a.numbers[1] = "OH NOES"
> > $a.numbers
> 1 OH NOES 3
> 
> Notice that the attribute @.numbers is readonly (since it's not
> declared 'is rw'). But that does us a fat lot of good here; it just
> means that I'm not allowed to reassign the array reference itself. The
> contents are just as writable as any array's.
> 
> What worries me -- and the reason I turn to p6l for this -- is that a
> lot of class authors will fail to notice this fact and write code that
> isn't properly encapsulated and, depending on the circumstances, maybe
> even insecure.

This is not specific to Perl 6 at all, and is a well-known problem in
object-oriented languages.  Some quick playing with search engines
turns up the phrase "defensive copy".

> Quoting from S06:639, which talks about readonly routine parameters:
> 
> ] By default, all parameters are readonly aliases to their corresponding
> ] arguments--the parameter is just another name for the original
> ] argument, but the argument can't be modified through it.  This is
> ] vacuously true for value arguments, since they may not be modified in
> ] any case.  However, the default forces any container argument to also
> ] be treated as an immutable value.  This extends down only one level;
> ] an immutable container may always return an element that is mutable if
> ] it so chooses.
> 
> Applying this semantics to readonly attributes would make the above
> issue go away. The problem with the semantics is that there doesn't
> seem to be a way to implement it that is both performant and free from
> surprising side effects.

To me it is obvious that non-rw attributes, non-rw parameters, and non-rw
block returns should use the same semantics.  Of course that leaves the
question of what those semantics should be.

> I solved this in my code by writing my own accessor that clones the array:
> 
> $ perl6
> > class A { has @!numbers; method add($n) { @!numbers.push($n); return }; 
> > method numbers { @!numbers.clone } }
> > my $a = A.new
> > $a.add(1); $a.add(2); $a.add(3)
> > $a.numbers[1] = "OH NOES"
> > $a.numbers
> 1 2 3
> 
> It felt kinda silly to write that accessor. Note also that I had to
> explicitly 'return' from the .add method, because .push returns the
> array acted on, and that would otherwise have been return, and the
> class would again be vulnerable. So it's not just accessors that are
> the problem.

You didn't have to add that return.  Method add is not declared
C, which means that its return value is automatically
protected against modification in the same sort of way as a non-rw
attribute.

> But it gets worse.
> 
> $ perl6
> > class B { has %!info = foo => { bar => 42 }; method info { %!info.clone } }
> > my $b = B.new
> > $b.info.perl
> {"foo" => {"bar" => 42}}
> 
> > $b.info = "OH NOES"
> Cannot modify readonly value
> > $b.info = 42
> > $b.info.perl
> {"foo" => {"bar" => 42}}
> 
> > $b.info = "OH NOES"
> > $b.info.perl
> {"foo" => {"bar" => "OH NOES"}}
> 
> We've done the cloning, so we're safe from changes on the shallow
> level. But changes on the levels below go through. (Same goes for
> arrays?) Why? When we clone the hash, we copy over the keys/values to
> a new hash. The value in this case is a hash reference, so the new
> hash gets the same hash reference.
> 
> I ended up writing a custom deep-cloner for hashes in order not to
> leak any information from the class:
> 
> $ perl6
> > sub deepclone(%h) { hash map -> $k, $v {; $k => ($v ~~ Hash ?? 
> > deepclone($v) !! $v ) }, %h.kv }
> > class B { has %!info = foo => { bar => 42 }; method info { 
> > deepclone(%!info) } }
> > my $b = B.new
> > $b.info = "OH NOES"
> > $b.info.perl
> {"foo" => {"bar" => 42}}
> 
> This solves it, but in a specialized and ad-hoc way.
> 
> The whole thing leaves me with the following questions:
> 
> * What do we mean when we say 'has @.a is readonly'? What do we want it to 
> mean?

It means has @!a; method a() { @!a }.  @!a is returned in a way that is
readonlyified in accordance with the rules of non-rw blocks.

> * Are we fine with references from readonly attributes leaking out of
> the class? Should we be?

I am, personally.  It's not a problem unique to Perl 6 and I have yet to
see a solution (in any language) that fails to be worse than the original
problem.

> * What language components could be provided to put class implementors
> on t

eval and try should be separate

[Stefan O';Rear]

I intend to change the definition of "eval" such that it does not catch
exceptions.  String eval's role as the catcher of exceptions is a legacy of
Perl 1, which had no block eval, and I feel it has no place in Perl 6.

The exception catching and associated unwinding makes it impossible to use
resumable exceptions across eval boundaries.  This includes warn.

Given an eval that does not catch exceptions, it is very easy to add catching,
using the new blockless form of try, "try eval $code".  However, given an eval
which does catch, it is impossible to synthesize one that passes exceptions
faithfully.

Catching exceptions in an eval causes the eval frame to be different from the
calling frame, which makes tail call optimization through evals impossible.

With the catching eval, it is very easy to write code which accidentally
discards exceptions.  For the goal of safety it seems best to make discarding
exceptions hard by default.

Does anyone have objections?  Is there general consensus that this change
should be made?

-Stefan
diff --git a/S29-functions.pod b/S29-functions.pod
index 21a0cc4..9459036 100644
--- a/S29-functions.pod
+++ b/S29-functions.pod
@@ -226,7 +226,9 @@ for C<$lang> would usually do to input files.
 The default for C<$lang> is the language in effect at the exact
 location of the eval call.
 
-Returns whatever C<$code> returns, or fails.
+Returns whatever C<$code> returns.  If C<$code> generates an exception,
+that exception will be propagated, unlike Perl 5.  Use "try eval" if
+you want to catch exceptions.
 
 =item evalfile
 


signature.asc
Description: Digital signature

Re: eval should throw an exception on compile error

[Stefan O';Rear]

On Sat, May 07, 2011 at 03:45:02PM +1000, Michael G Schwern wrote:
> I was just playing around with eval, trying to figure out if you can define an
> operator overload at runtime (seems you can't, good) and noticed this in the
> spec... [1]
> 
> "Returns whatever $code returns, or fails."
> 
> How does one get the compile error from an eval?  What's the equivalent to $@?
>  I don't see anything in the eval tests [2] that's checking for the error,
> just that it returned false.

The error goes into $!, which is automatically lexical.

What you're seeing is a bug-of-omission in Rakudo; if $! is not checked by
the end of the scope, Perl 6 is supposed to raise an immediate exception.

> In addition, I'm surprised that eval doesn't throw an error when it fails.
> 
> try { eval $code; CATCH { ... } }
> 
> Using eval STRING in Perl 5 is pretty rare, and should hopefully be rarer in
> Perl 6 than Perl 5.  While it can be used as a "does this compile" check, it's
> overwhelmingly used with the expectation that the code will compile and so it
> is an exception if it does not.  More pragmatically, this resolves the problem
> of $@ by putting it into an exception rather than a side global.
> 
> It also resolves this trap:
> 
> if eval $code {
> say "code returned true";
> }
> else {
> # this will also happen if $code fails to compile
> say "code returned false";
> }
> 
> While Perl 6 probably has some sort of magic "I failed" value you can check
> for separate from boolean false, folks aren't going to go through the extra
> convolutions.

Larry doesn't like immediate exceptions very much, citing something about
"parallel processing".  fail() is much preferred for stuff in the specs.

-Stefan


signature.asc
Description: Digital signature

Enumerating prefix operators, and spec changes

[Stefan O';Rear]

[In response to a comment on perl6-compiler, in the thread
"[perl #85746] spec/S29-context/sleep.t is way too relax"]

On Wed, Mar 09, 2011 at 12:31:34PM -0600, Patrick R. Michaud wrote:
> While researching this ticket, I notice that S29 has a C 
> function defined (line 246); this seems to be in contradicition 
> with S03 which defines a prefix: operator.  This 
> contradiction likely needs resolving.

This is a more general problem: the distinction between prefix ops
and functions is not well defined.  For instance, 'defined 1 && 0'
is parsed as defined(1) && 0 by Rakudo, but as defined(1 && 0) by
STD and derivatives.

Who has the authority to make changes to the spec like this, that
would invalidate existing programs for the sake of a perceived
improvement?  I would change this, and a few other things, but feel
I lack this authority.  I wonder what the correct process is.

-sorear


signature.asc
Description: Digital signature

Re: IO Multiplexing

[Stefan O';Rear]

On Fri, Nov 12, 2010 at 06:59:59PM -0800, Ben Goldberg wrote:
(snip plea to paint the bikeshed fuchsia)

The design of the I/O system will be chosen by the first person to implement
it.  If you want any say in the matter, you need to be that person.  Bonus
points if you also port at least one app to prove that your system is usable.

-sorear


signature.asc
Description: Digital signature

Re: Packed arrays and assignment vs binding

[Stefan O';Rear]

On Sat, Nov 13, 2010 at 06:09:00PM +0100, Jonathan Worthington wrote:
...
> With packed arrays, however, I'm less clear what they mean. Since
> the point of a packed array is compact storage, there's no chance to
> actually have containers. Thus does assignment to a slot in a
> compact array ever make sense? There's not a container to look up
> and store things in.
> 
> While I can happily convince myself that:
> 
> @x[0] := 1; # works, just sticks 1 into the appropriate location
> @x[0] = 1; # dies, can't assign when there's no container
> 
> Actually makes sense, I can also somewhat see users not liking it.
> So my questions are:
> 
> 1) What semantics would users expect? Is it OK to say "no, you can't
> assign" in this case?
> 
> 2) If proposing that both should work, what, roughly, would that
> look like at an implementation level?

I have already come up with exactly the same conclusion on my own.  +1

Also, given "my int $x", "$x := 5" should work, but not "$x = 5".

-sorear


signature.asc
Description: Digital signature

Re: IO Multiplexing

[Stefan O';Rear]

On Thu, Nov 11, 2010 at 05:47:46PM -0800, Ben Goldberg wrote:
> I would like to know, is perl6 going to have something like select
> (with arguments created by fileno/vec), or something like IO::Select
> (with which the user doesn't need to know about the implementation,
> which happens to be done with fileno/vec/select), or only an event
> loop.

I will be unhappy if Perl 6 doesn't provide all three.  However, the
spec in question doesn't really exist.  The "IO" synopsis is garbage
and should be deleted and rewritten from scratch by someone who knows
what they're talking about.

> I would recommend that there NOT be any sort of fileno exposed to the
> user, unless he goes out of the way to get it -- any function (in
> particular, posix functions) should simply take a perl filehandle, and
> that function in turn would pull out the fileno (or fail
> appropriately, if the filehandle doesn't have a fileno).  If users
> want to know if filehandles correspond to the same underlying file,
> then there could be a method -- perhaps $fh.uses_same_desciptor($fh2),
> or somesuch.

This goes without saying.  One caveat - it should be possible to pass
integer file descriptors.

> If there's a select() builtin (and I'd much rather that there not be
> -- it should be hidden away in a class, like perl5's IO::Select), I'd
> very much hope that it would take and return Sets of filehandles, not
> vec packed strings.  I'd prefer there not be one[**]

Absolutely not.  TIMTOWDI.  There will be a select() function (in
the POSIX module, not the default namespace!), and it will take
parameters as close as possible to POSIX.1's definition.  Perl without
system calls is not Perl.

I'm totally fine with discouraging casual use, though, which is why
it shouldn't be in the default namespace.

> If there's something like perl5's IO::Select, it should be able to
> "just work" regardless of whether the perl filehandles are sockets,
> regular files, or user-created pure-perl filehandles (which might
> never block, or which might use one or more normal filehandles
> internally, which in turn might potentially block).  This is what I'd
> prefer.

That is a good doctoral thesis topic.

> Lastly, if perl6 has an efficient enough built-in event loop, and
> sufficiently lightweight coroutines (or maybe I should say fibers?),
> then we might not need to have any kind of explicit multiplexing.

TIMTOWDI.  Perl without system calls is not Perl.

> For example, any time user code does a read operation on a handle that
> isn't (from the user code's point of view) in nonblocking mode, the
> filehandle implementation would tell the the event loop to yield to it
> when the handle becomes readable, then it would yield to the event
> loop, then (once it gets back control) read from the handle.[*]

This is why S16 is junk - too much blue-sky thinking, not enough
pragmatism and practical experience.

> This provides lots of convenience, but it would resemble Java IO
> before the NIO -- except with one fiber per handle instead of one
> thread per handle.  Coroutines/green threads/fibers are much lighter
> weight than "real" threads, but often aren't as fast as a well-written
> select() loop specially written for the user's task.
> 
> Thus, I'd hope for perl6 to have an IO::Select, and automatically-
> yielding [*] blocking IO, and not have a select() builtin.  [**]

TIMTOWDI.

> [*] This is a simplification:
> A) If a user explicitly marks a filehandle as not yielding to other
> coroutines, it would do a blocking read (or whatever) instead of going
> through the event loop rigmarole.
> B) If perl6 was compiled with an asynchronous IO library (or is on
> windows and is not using stdio and has (Read|Write)FileEX support),
> then it might start the Async IO operation, tell the event loop to
> wake it when the operation completes, then yield to the event loop.
> C) Depending on circumstances, it *may* be more efficient to have the
> event loop itself do the reading or other IO itself, and schedule the
> fibers for which the IO was done, than to have the fibers do the IO.
> TMTOWTDI.  This would be especially important if perl is compiled with
> async IO -- the event loop might first wait for the fds to be readable/
> etc, *then* start the async IO for those fds, then schedule the fibers
> for which the performed IO has completed, thus minimizing the number
> of outstanding async io operations.
> 
> [**] The main reason I'd prefer that perl6 not have a select() builtin
> is that every time it's called perl would need to convert user-level
> Sets of filehandles into the underlying implementations' versions of
> them (fd_sets on unixy, fd_sets and/or an event queue handle on
> windows), and then back to perl Set objects, and free up the
> implementation version of the filehandle set... this is inefficient.
> 
> A well written IO::Select-like object could create (potentially empty)
> versions of the OS's set of filehandles when it's created, add to that
> set as nee

Re: Ruby Fibers (was: threads?)

[Stefan O';Rear]

On Fri, Oct 15, 2010 at 01:42:06PM +0200, Leon Timmermans wrote:
> On Wed, Oct 13, 2010 at 1:20 AM, Tim Bunce  wrote:
> > I've not used them, but Ruby 1.9 Fibers (continuations) and the
> > EventMachine Reactor pattern seem interesting.
> 
> Continuations and fibers are incredibly useful and should be easy to
> implement on parrot/rakudo

Forget Parrot, fibers can be implemented in pure Perl 6.

module Fibers;

my @runq;

sub spawn(&entry) is export {
push @runq, $( gather entry() );
}

sub yield() is export {
take True;
}

sub scheduler() is export {
while @runq {
my $task = shift @runq;
if $task {
$task.shift;
push @runq, $task;
}
}
}

-sorear


signature.asc
Description: Digital signature

Re: Breaking encapsulation by detaching a private-variable-accessing method from one object and calling it on another

[Stefan O';Rear]

On Sat, Jul 31, 2010 at 02:36:02PM -0400, Brandon S Allbery KF8NH wrote:
> On 7/31/10 14:23 , Carl Mäsak wrote:
> > a. Allow this form of encapsulation breakage.
> > b. Disallow detaching of certain methods.
> > c. Disallow attaching of certain anonymous methods.
> > 
> > I must confess I don't particularly like either option. I'm by no
> > means an OO expert. It would be interesting to hear your views on
> > this.
> 
> The whole concept of detaching and attaching methods seems suspect to me; in
> particular, attaching a method from a class not declared to be related reeks
> of monkey patching.  As such, I'd only allow it when monkey patching is 
> enabled.

Methods are just functions.

$object.$method(@args)

is simply sugar for

$method($object, @args)

so disallowing it is not quite that simple.

-sorear


signature.asc
Description: Digital signature

Re: Breaking encapsulation by detaching a private-variable-accessing method from one object and calling it on another

[Stefan O';Rear]

On Sat, Jul 31, 2010 at 08:23:29PM +0200, Carl Mäsak wrote:
> * It has been decided that attribute slots of the type $!foo are only
> allowed *syntactically* within the class block that declares them.
> (The exception to this, I guess, is the 'trusts' directive.) But this
> means that something like this anonymous method
> 
> my $reveal-foo = method { say $!foo }
> 
> isn't allowed. I think that's good, because it would provide a very
> easy way to break encapsulation of an object; just call
> $object.$reveal-foo() on it.

There is no $!foo.  There is only $!Class::foo, and $!foo is a lexically
scoped alias to it.  This is necessary to allow privacy from your children
to work:

class Class {
has $!foo;  # the mere existance of $!foo is an implementation detail
}

class SubClass is Class {
has $!foo;  # hey why doesn't this work?
}

So $reveal-foo can't be defined because $!foo isn't even in scope.

> * Today we discovered that it's possible to break encapsulation by
> detaching a method from an object of one class, and calling that
> method on an object of another class. Which means that breaking the
> encapsulation of a foreign class is as easy as creating a custom class
> with all of the same private attributes, and with a method to print
> (or otherwise reveal) them.

Calling such methods should fail, because the $!OtherClass:: attributes
don't exist even if the shortnames are the same.

> * It is my feeling that such encapsulation-breakage shouldn't be
> allowed. Do you agree, p6l?

It's not.

> * If it isn't allowed, which of the two steps is disallowed?
> *Detaching* a method containing references to private accessor slots
> (thereby extending the syntactic restriction of "no private accessors
> outside of the class block"), or *attaching* an anonymous method to an
> object belonging to a different class than the one from which it was
> detached (thereby by necessity having to complicate anonymous methods
> somewhat)?
> 
> I only see those three options:
> 
> a. Allow this form of encapsulation breakage.
> b. Disallow detaching of certain methods.
> c. Disallow attaching of certain anonymous methods.
> 
> I must confess I don't particularly like either option. I'm by no
> means an OO expert. It would be interesting to hear your views on
> this.

Perl philosophy has always been "it's not that I have a shotgun, you just
weren't invited, so stay out of my living room".  I don't see any reason for
Perl 6 to break with this - encapsulation should be about helping the
programmer, not helping the sysadmin maintain system security.

-sorear


signature.asc
Description: Digital signature

Re: Announce: Rakudo Perl 6 development release #30 ("Kiev")

[Stefan O';Rear]

On Thu, Jun 17, 2010 at 04:55:38PM -0700, Darren Duncan wrote:
> So, is "Rakudo Star" meant to be a parallel release series, sort of like 
> Perl 5.12.x vs 5.13.x are now, or are the monthly Rakudo releases we've 
> been seeing going to be named "Star" at some point?  I thought I read 
> recently that "Star" would be coming in June. -- Darren Duncan

Rakudo Star is a parrallel release series; however, it is not a series of
compiler releases, but a series of complete Perl 6 environments.  A
single release of Rakudo Star will contain:

- Some version of Rakudo (not necessarily a monthly)
- Some version of Parrot
- Some random libraries
- Documentation

all vetted to work OK together.

-sorear


signature.asc
Description: Digital signature

Re: Reversible grammars

[Stefan O';Rear]

On Sat, Jun 05, 2010 at 09:19:01AM +1000, Timothy S. Nelson wrote:
>   Hi.  I've been thinking more about reversible grammars.  Specifically,  
> I'm wondering if the following pseudo-code will be possible:
>
> ## Match a grammar here
> $match = Grammar.match($text)
> ## Need some code here to get $submatch from $match
> 
> $submatch.Str = "fred"
> ## Reverse Grammar
> $text = Grammar.reverse($match)
>
>   My idea is that I could read in a file, find the part I wanted in the  
> match tree, change that part, and then write it back to the file.  Is 
> that going to be a possibility?

Look at emit_p5 in viv, where we do basically exactly that (but combined with
a recursive traversal).

>   Also, while I was trying to investigate this, I was wondering about  
> S32/Rules.  I've added a few more lines to it, but there are some things 
> I'm having trouble figuring out (this may be due to a few months absence 
> from the p6 community -- sorry :) ).  Anyway, my questions are:

Higher-numbered synopses are much more speculative, poorly implemented, and
get little love from the janitorial staff; so, in case of disagreement, the
lower number has the correct answer.

> - I note that, while S32/Rules says that .from returns an Int, S05 has a
>   call $/.from.bytes .  Does this mean that there's a StringSize type

S32 is wrong.

>   that does maybe .bytes, .chars, .graphs, and .codes that's returned by
>   maybe .from, .to, and .pos ?

Yes, it's called StrPos; see S02:813.

> - I've also noticed that there's a .chars method on the match object;
>   wouldn't this exist by default because of the Str method?

-sorear


signature.asc
Description: Digital signature

Re: r31082 -[S32/Str] rethinking of tab characters

[Stefan O';Rear]

On Thu, Jun 03, 2010 at 07:00:17PM +0200, Carl Mäsak wrote:
> sorear (>):
> > 2. Indenting a blank line results in a blank line, not a line with only
> >   whitespace.
> 
> What about indenting a line with only whitespace?

Implementor's choice; it won't come up in the viv port.

> I think I can see use cases both for special-casing and for not
> special-casing indenting empty lines. Perhaps that indicates that
> this, too, should be a flag. :indent-empty-lines or something?

Agreeable.

> > 3. A variant of indent be provided which does not treat the first character
> >   specially, perhaps named .indent(4 :hang) or .hang.
> 
> I fail to understand what 'does not treat the first character
> specially' means here. Do you have an example?

is("a\nb".indent(1), " a\n b", "indent puts space on the first line");
is("a\nb".hang(1), "a\n b", "hang does not put space on the first line");

hang is useful in formatting applications involving left-to-right composition,
such as "foo: " ~ $text.hang(5)

-sorear


signature.asc
Description: Digital signature

Re: r31082 -[S32/Str] rethinking of tab characters

[Stefan O';Rear]

On Thu, Jun 03, 2010 at 03:52:02PM +0200, pugs-comm...@feather.perl6.nl wrote:
> Author: masak
> Date: 2010-06-03 15:52:01 +0200 (Thu, 03 Jun 2010)
> New Revision: 31082
> 
> Modified:
>docs/Perl6/Spec/S32-setting-library/Str.pod
> Log:
> [S32/Str] rethinking of tab characters
> 
> Also added a Str.indent(*) use case.

I request that:

1. Blank lines should not be interpreted as having 0 indentation.  Instead,
   lines consisting entirely of horizontal whitespace should be ignored in
   indent(*) considerations, and can be unindented by any amount.  Unindenting
   a truly blank line has no effect.

2. Indenting a blank line results in a blank line, not a line with only
   whitespace.

3. A variant of indent be provided which does not treat the first character
   specially, perhaps named .indent(4 :hang) or .hang.

-sorear


signature.asc
Description: Digital signature

Re: optional rules cluttering parse trees

[Stefan O';Rear]

On Tue, Apr 27, 2010 at 06:31:01AM +0200, Stéphane Payrard wrote:
> When doing an analyse of a sample parse tree, I note that it is
> cluttered by the reduction of optional subrules
> to  generate a zero length parse subtree. That is, rules with a '?'
> quantifier matching zero time.
> Suppressing such matching rules from the parse tree would make it
> easier to read.

I suppose it would also make the parse tres considerably smaller.
I wonder how much of Rakudo's memory usage (currently 361 MiB peak
in the setting) is caused by parse trees, and how much this
optimization would help?

-sorear


signature.asc
Description: Digital signature

Re: Methodicals: A better way to monkey type

[Stefan O';Rear]

On Wed, Apr 21, 2010 at 12:35:11PM -0300, Daniel Ruoso wrote:
> Em Qua, 2010-04-21 às 00:16 -0700, Stefan O'Rear escreveu:
> > Normally, when you write a method call, the definition of the method is
> > entirely in the domain of the receiver's class:
> > $object.make-me-a-sandwich;  # $object gets to decide what this means
> 
> Actually, this is delegated to the dispatcher, which is composed in at
> least three different parts:
> 
>   1 - The general dispatching mechanism which iiuc, is lexically 
>   defined and which, by default, implements something like:
> 
> @*current_invocation_candidates = $object.^can($capture);
> @*current_invocation_candidates.shift.postcircumfix:<( )>($capture);
> 
>   2 - The MRO resolver, which is implemented inside ^can, and is
>   completely internal to the class.
> 
>   3 - The actual routine invocation, which is internal to the actual 
>   method object.

I failed to find this in the spec, would you give a reference?  It looks
very interesting.

> > However, this is not always how things work.
> > $object.WHAT;  # Larry says that WHAT should return the protoobject
> > WHAT, WHENCE, postcircumfix:<[ ]>, and similar operators have semantics
> > (although not implementation) defined by the language itself.
> 
> Actually WHAT, WHENCE and the like are considered macros, not method
> calls.

S03 calls WHAT et al macros, S02 calls them methods.

This proposal is somewhat related to macros, in that it can be emulated with
them, much as you can emulate user-defined infix operators by macroing them
to ordinary subs.

> Postcircumfix:<[ ]> is actually an operator, not a method, so they are
> always lexically dispatched.

Treating syntactic categories as subs regardless of call syntax is cruft which
could be eliminated with a more general system.

> > Currently, Perl 6 fakes this by having all values inherit from Any or
> > Mu or Cool, which define the basic, overridable versions of these
> > method-like operators.
> 
> Hmm... I never thought of it as a fake, it's a simple fact, every value
> in Perl 6 should derive from Mu, every non-special type is derived from
> Any.
> 
> > This cheat actually works pretty well in a homogeneous environment,
> > but it fails if we have to consider objects from outside Perl 6.
> > $parrot-array.WHAT;  # Method WHAT not found for invocant of type ...
> 
> This is actually a rakudobug, a $parrot-array should be properly boxed
> into a value that provides a protoobject... Again, .WHAT is not a
> method.

Autoboxing at language boundaries works very badly in the real world.  I
speak as a Blizkost maintainer, where autoboxing is technically required.
I would hate to see Rakudo follow me in the trail of tears.

> > The problem here is that scopes are conflated - WHAT is defined by the
> > language, which is approximately a lexical scope, while methods on an object
> > are in some entirely different scope.
> 
> AFAIU, method dispatch is also lexically controlled - to some extent.

I'd love a Sxx: on that, sounds like an interesting section.

> > A methodical is an operator which syntactically behaves as a method but is
> > subject to scoping rules.  Methodicals are defined using the ordinary method
> > keyword, qualified with my or our.  (TODO: This seems the most natural 
> > syntax
> > to me, but it conflicts with existing usage.  Which is more worth having on
> > it?)  Methodicals do not need to be declared in classes, but they should
> > generally have declared receiver types.
> > Thoughts?
> 
> Apparently you're trying to override the default dispatching mechanism
> which, I think, is something already supported by Perl 6 (although not
> yet fully spec.

Then perhaps this could serve as the start of a spec.

-Stefan


signature.asc
Description: Digital signature

Methodicals: A better way to monkey type

[Stefan O';Rear]

(The following describes a proposed extension to Perl 6, methodical scoped
operators or methodicals for short.  It is written in the present tense for
simplicity but should be understood as the future conditional.)

Normally, when you write a method call, the definition of the method is
entirely in the domain of the receiver's class:

$object.make-me-a-sandwich;  # $object gets to decide what this means

However, this is not always how things work.

$object.WHAT;  # Larry says that WHAT should return the protoobject

WHAT, WHENCE, postcircumfix:<[ ]>, and similar operators have semantics
(although not implementation) defined by the language itself. Currently, Perl 6
fakes this by having all values inherit from Any or Mu or Cool, which define
the basic, overridable versions of these method-like operators. This cheat
actually works pretty well in a homogeneous environment, but it fails if we have
to consider objects from outside Perl 6.

$parrot-array.WHAT;  # Method WHAT not found for invocant of type ...

The problem here is that scopes are conflated - WHAT is defined by the
language, which is approximately a lexical scope, while methods on an object
are in some entirely different scope.

A methodical is an operator which syntactically behaves as a method but is
subject to scoping rules.  Methodicals are defined using the ordinary method
keyword, qualified with my or our.  (TODO: This seems the most natural syntax
to me, but it conflicts with existing usage.  Which is more worth having on
it?)  Methodicals do not need to be declared in classes, but they should
generally have declared receiver types.

{
my method make-me-a-sandwich(Num:) { ... }

2.make-me-a-sandwich; # calls our method

"ham".make-me-a-sandwich; # either an error, or method dispatch
}

2.make-ma-a-sandwich; # ordinary method dispatch

The primary use case for methodicals is naturally in the setting.  Many of the
methods which are currently defined on lower-level types could become
methodicals and they would be available on all values, in any code compiled
within the Perl 6 setting, without polluting the referencing environments of
non-Perl 6 code in the same object environment.

my method WHAT($thing) { ... }

Rakudo already uses something very much like an ad-hoc direct implementation of
methodicals to handle postcircumfix operators.

Methodicals are also useful in user code where the most natural phrasing of a
calculation is as a property of a system type, for instance from the Mandelbrot
Advent example:

my method mandel(Complex:) {
my $z = 0i;
for ^$max-iterations {
$z = $z * $z + self;
return 1 if ($z.abs > 2);
}
return 0;
}

Since methodicals do not pollute global referencing environments, and do not
compromise encapsulation, MONKEY_TYPING is not needed here.

A methodical defined inside a class functions much like a private method.
There may be something interesting here, but I'm not entirely sure what.

There are two natural implementations of methodicals in terms of other language
features.  First, it is possible to treat methodicals as ordinary methods
(monkey typing under the hood), but with generated names; the true name is
bound to the program name only within the correct scope.  If possible, the
methods should be treated as anonymous, accessible only though an opaque name
object.  Care must be taken to prevent incorrect exposure of private fields and
methods.

Alternatively, a methodical can be desugared to a multi sub with the same scope
as the methodical itself, changing statically named method calls into calls to
the methodical.  This gives reflection invisibility and privacy for free, but
may require more work to get the multi call semantics exactly right.

Thoughts?

-Stefan


signature.asc
Description: Digital signature