Re: coded size limits on Perl data types?

[Darren Duncan]

Thank you for this Timo, and to everyone else who replied.  It did indeed 
address what I wanted to know. -- Darren Duncan


On 2016-09-13 5:15 AM, Timo Paulssen wrote:

I'll answer based on the data structures MoarVM uses internally:

On 09/13/2016 05:13 AM, Darren Duncan wrote:> (Pretend the actual hardware has
infinite memory so we wouldn't run > out of hardware first.) > > 1.  Maximum
size of a (non-machine) integer? We're using libtommath, which declares the
"used" and "alloc" fields of the mp_int as "int", iow a 32bit signed (???) 
integer.


2.  Maximum number of elements in an array?  MVMArray declares the "elems", "start" and 
"ssize" fields to be MVMuint64, so

they can become quite a bit bigger than strings.


3.  Maximum number of elements in a hash?  the "uthash" library we're using declares the 
"num_buckets" and "num_items"

fields as unsigned (so 32bit unsigned integer).


4.  Maximum number of bytes/codepoints/etc in a character string?  MVMString declares its 
"num_graphs" as a MVMuint32, but a graph can be

multiple codepoints and as such multiple bytes when encoded.


Following the above, does the Perl 6 language specify any such  > limits, or 
does it define the above things to be infinite? I don't think it

does.


Hope that helps!
   - Timo

Re: grammars and indentation of input

[Theo van den Heuvel]

As so often it turned out that the reason my program did not work was 
elsewhere (in the grammar).

My approach worked al along.
It was instructive to look at the examples you guys mentioned. Thanks

Theo

Re: Fwd: Re: grammars and indentation of input

[Bennett Todd]

Well put.

The clearest description of Python's approach I've read, explained it as a 
lexer that tracked indentation level, and inserted appropriate tokens when it 
changed.

Re: Fwd: Re: grammars and indentation of input

[Bennett Todd]

Hostile or not, thanks for your informative reply.

Re: Fwd: Re: grammars and indentation of input

[Bennett Todd]

Thank you, very much. Yes, I'm disappointed, but I'd rather know.

Re: Fwd: Re: grammars and indentation of input

[Aaron Sherman]

>
> Having the minutia of the programmatic run-time state of the parse then
> influence the parse itself, is at the heart of the perl5 phenomenon "only
> Perl can parse perl"


I don't mean to be hostile, but you're demonstrably wrong, here. (also it's
"only perl can parse Perl" as in, only the "perl" implementation can parse
the Perl language).

There are currently about 2.5 implementations of Perl 6, and while you
could backhandedly claim that only Perl 6 can parse Perl 6 (because it's
specced as a self-hosting language whose spec is actually written in Perl
6), the reality is that the parts that aren't written in Perl 6 can be
written in just about anything (with C/MoarVM and JVM implementations
working just fine).

It's not a context sensitive grammar that was the issue with Perl 5, it was
the lack of a specification outside of the primary implementation.



Aaron Sherman, M.:
P: 617-440-4332 Google Talk, Email and Google Plus: a...@ajs.com
Toolsmith, developer, gamer and life-long student.


On Tue, Sep 13, 2016 at 10:35 AM, Bennett Todd 
wrote:

> Having the minutia of the programmatic run-time state of the parse then
> influence the parse itself, is at the heart of the perl5 phenomenon "only
> Perl can parse perl", which I rather hope isn't going to be preserved in
> perl6.
>

Re: Fwd: Re: grammars and indentation of input

[Patrick R. Michaud]

On Tue, Sep 13, 2016 at 10:35:01AM -0400, Bennett Todd wrote:
> Having the minutia of the programmatic run-time state of the parse then 
> influence the parse itself, is at the heart of the perl5 phenomenon "only 
> Perl can parse perl", which I rather hope isn't going to be preserved in 
> perl6.

You may be disappointed to read this:  Not only is this feature preserved in 
Perl 6... it's something of a prerequisite.  It is what is required for a truly 
dynamic language that has many things happening in BEGIN blocks (i.e., things 
get executed even before you finish compiling the thing you're working on) and 
that allows dynamically adding new statement types and language features to the 
grammar.

When implementing Perl 6, I think many of us aimed to minimize the amount of 
"runtime" things that happened during the parse... only to discover that we 
actually had to embrace and formalize it instead.

Pm

Re: Fwd: Re: grammars and indentation of input

[Bennett Todd]

Having the minutia of the programmatic run-time state of the parse then 
influence the parse itself, is at the heart of the perl5 phenomenon "only Perl 
can parse perl", which I rather hope isn't going to be preserved in perl6.

Re: grammars and indentation of input

[Aaron Sherman]

I don't see why optimization would frustrate this approach. You are doing
the correct thing as far as I can tell, but with one exception. The current
implementation (last I checked) was sometimes slow in binding values. You
might need to force it between an assignment and passing a bound match as a
parameter by inserting an empty block. You can see this documented and used
here:

http://examples.perl6.org/categories/parsers/SimpleStrings.html



Aaron Sherman, M.:
P: 617-440-4332 Google Talk, Email and Google Plus: a...@ajs.com
Toolsmith, developer, gamer and life-long student.


On Tue, Sep 13, 2016 at 7:13 AM, Theo van den Heuvel 
wrote:

> Hi all,
>
> I am beginning to appreciate the power of grammars and the Match class.
> This is truly a major asset within Perl6.
>
> I have a question on an edge case. I was hoping to use a grammar for an
> input that has meaningful indented blocks.
> I was trying something like this:
>
>   token element { <.lm> [  | $=[ ' '+ ]  ] }
>   token lm { ^^ ' '**{$cur-indent} } # skip up to current indent level
>
> My grammar has a method called within the level rule that maintains a
> stack of indentations and sets a $cur-indent.
> I can imagine that the inner workings of the parser (i.e. optimization)
> frustrate this approach.
> Is there a way to make something like this work?
>
> Thanks,
> Theo
>
> --
> Theo van den Heuvel
> Van den Heuvel HLT Consultancy
>

Re: coded size limits on Perl data types?

[Timo Paulssen]

On 09/13/2016 03:12 PM, Timo Paulssen wrote:> If one big integer is
allowed to be 14 gigabytes big (if we use the > default of 28 bits per
"mp digit"; it's also possible to use 31 or > 60.), we can still safely
say "limited only by memory" for now.

Something else that makes this pretty difficult is that our big ints
currently act immutable, so if we ++ a number all the way up to 14
gigabytes, we'll keep around a whole bunch of older big ints until the
GC thinks it has to kick in ...

If you have an exabyte of RAM or a significant amount of swap space, you
ought to be able to work with this, though.

Be prepared for The Slow, though :)

Re: grammars and indentation of input

[Brian Duggan]

I've also recently been experimenting with parsing an
indent-based language -- specifically, a small subset
of Slim () -- I push to a stack
when I see a tag, and pop based on the depth of the
indendation.

Here's a working example:

  https://git.io/vig93

Brian

Re: coded size limits on Perl data types?

[Elizabeth Mattijsen]

> On 13 Sep 2016, at 14:15, Timo Paulssen  wrote:
> 
> I'll answer based on the data structures MoarVM uses internally:
> 
> On 09/13/2016 05:13 AM, Darren Duncan wrote:
> 
> > (Pretend the actual hardware has infinite memory so we wouldn't run
> > out of hardware first.)
> > 
> > 1.  Maximum size of a (non-machine) integer?
> 
> 
> We're using libtommath, which declares the "used" and "alloc" fields of the 
> mp_int as "int", iow a 32bit signed (???) integer.

eh, but Int is supposed to be a bigint, only limited by memory, no?


Liz

Re: grammars and indentation of input

[Timo Paulssen]

I haven't read your code, but your question immediately made me think of
this module:

https://github.com/masak/text-indented

Would be interested to hear if this helps you!
  - Timo

grammars and indentation of input

[Theo van den Heuvel]

Hi all,

I am beginning to appreciate the power of grammars and the Match class. 
This is truly a major asset within Perl6.


I have a question on an edge case. I was hoping to use a grammar for an 
input that has meaningful indented blocks.

I was trying something like this:

  token element { <.lm> [  | $=[ ' '+ ]  ] 
}

  token lm { ^^ ' '**{$cur-indent} } # skip up to current indent level

My grammar has a method called within the level rule that maintains a 
stack of indentations and sets a $cur-indent.
I can imagine that the inner workings of the parser (i.e. optimization) 
frustrate this approach.

Is there a way to make something like this work?

Thanks,
Theo

--
Theo van den Heuvel
Van den Heuvel HLT Consultancy