Re: Compiling to bytecode

[Alexander Burger]

On Sun, Jul 16, 2017 at 10:17:40AM -0400, Matt Wilbur wrote:
> The MIPS in this case has 32 registers, 30 of which are general
> purpose (though there is some convention, as you might expect). r0 is
> hard coded to 0 and r31 is the stack pointer.

Sounds good. You need 12 for the 6 VM registers A,C,E and X,Y,Z. L and S may be
in a 32-bit register each, pointing into the stack. Then some more for temporary
stuff.


> I will have to look at the arm64 / ppc64 - do you assign machine
> register for each VM register?

Better, but you could as well put some of them into global memory. Probably not
necessary here.


> there any VM details published?

Only doc64/asm, and the sources in src64/arch/ I think.

♪♫ Alex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Alexander Burger]

On Sun, Jul 16, 2017 at 08:04:53AM -0400, Matt Wilbur wrote:
> Reality just intervened - my MIPS processor is native 32-bit.  So I think
> emu is my only option.  :(

Well, it would also be possible to generate code for a 32-bit machine, using two
physical registers for a single pil64-VM register. It is a bit of waste.

I thought about that initially for i386 - before writing emu - but felt that
x84-32 does not have enough registers to do it efficiently.

Would Mips32 have enough registers?

♪♫ Alex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Matt Wilbur]

On Sun, Jul 16, 2017 at 7:44 AM, Matt Wilbur  wrote:
> On Sun, Jul 16, 2017 at 7:26 AM, Alexander Burger  
> wrote:
>> On Sat, Jul 15, 2017 at 03:02:02PM -0400, Matt Wilbur wrote:
>>> Correct.  I am working on a project that uses a MIPS processo embedded
>>> ...
>>> I need the 64-bit PicoLisp, but MIPS isn't one of the
>>> architectures currently supported.
>>
>> Now I understand.
>>
>>> I started to look at how things
>>> are done for ARM and intel, but don't have the time right now to
>>> properly add MIPS.
>>
>> This is the right place. A MIPS port would probably be similar to the arm64 
>> and
>> ppc64 versions. But it is indeed a nasty piece of work. Each of the existing
>> ports took me several weeks. Funny thing is that the most tedious part was
>> always the floating point support (despite PicoLisp does not have floats in 
>> the
>> language, it must support them on the VM level for 'native' calls).
>
> I would very much like to take a crack at as I think it would be a
> great learning experience.  At a first glance the code that generates the
> assembly looks very reminiscent of some of the old assemblers written
> in Forth I once admired.  I am by no means a MIPS guru but I have
> enough resources I think I can figure stuff out.

Reality just intervened - my MIPS processor is native 32-bit.  So I think
emu is my only option.  :(


>
>>> So, I got the 64-bit PicoLisp compiled in emulator mode, after
>>> cross-compiling sysdefs, capturing the output in a text file, and then
>>> using that output in places where the output from sysdefs was read via
>>> a pipe.
>>
>> OK, good. The drawback with emu is the slow execution speed though.
>>
>>
>>> I had assumed (perhaps incorrectly) that, using emu, VM bytecode was
>>> created on the fly and that it gets "executed"
>>
>> The bytecodes (if we may call 16-bit words "bytes") are created at build 
>> time,
>> as you will have noticed, in the generated C source files.
>>
>>
>>> I am completely open to the idea that I am being completely wrong
>>> headed about something.
>>
>> Not wrong at all. The problem is only the missing MIPS port ;)
>>
>> ♪♫ Alex
>>
>> --
>> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[George Orais]

Hi Matt,

Nice plan you got!

Maybe I can share you some of my experience implementing PIL64 to FPGA.

But got some questions first:
1. Is it a 32bit or a 64bit MIPS?
2. Is it running an OS?

I'm not sure but I think someone was attempting to port PicoLisp to an old SGI? 
And I'm sure that SGI was using MIPS, maybe that port might help you?


BR,
Geo

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Matt Wilbur]

On Sat, Jul 15, 2017 at 7:21 AM, Jakob Eriksson  wrote:
> Still, PicoLisp source, I mean the program you write in PicoLisp,
> (not the C code) can become quite large depending on your application.
>
> We mitigated this, not by storing the "compiled" structures of pointers
> to pointers. (Although we could have - but that would have been
> complicated to implemented I think, and I am not convinced it would have
> been much smaller than the text source.)
> Instead, we compressed the text files (picolisp source code) and
> unpacked them upon on loading. Source code in general compresses very well.
> In principle, I think you could create something like these Javascript
> "minifiers" before putting it through compression. This hypothetical
> PicoLisp minifier would for instance replace long variable names with
> shorter ones.

This is another approach I that crossed my mind too, though I have
only passing interaction with the JS world.
I am assuming one could implement a load-like function that uses gzip
and decompresses the stream in addition to reading.

Matt
>
>
> https://github.com/mishoo/UglifyJS
>
>
>
>
>
>
>
> On 2017-07-15 12:56, Alexander Burger wrote:
>> Hi Matt,
>>
>>> In order to minimize flash footprint, I was wondering if it would be
>>> possible to compile code down to byte code?  The reason i wanted to do that
>>> is then I could make of some library functions in reader macros off-target
>>> and avoid installing libraries on the target.
>>
>> Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
>> binary, right?
>>
>> Note that normal PicoLisp doesn't compile anything. It "compiles" to 
>> structures
>> of pointers to pointers, where the compiler is called the "reader" in Lisp.
>>
>> As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
>> source files to C structures, to be able to put them into ROM instead of
>> precious RAM at runtime. However, these are still not byte codes, but the 
>> good
>> old pointers to pointers :)
>>
>> ♪♫ Alex
>>
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Matt Wilbur]

On Sat, Jul 15, 2017 at 6:56 AM, Alexander Burger  wrote:
>
> Hi Matt,
>
> > In order to minimize flash footprint, I was wondering if it would be
> > possible to compile code down to byte code?  The reason i wanted to do that
> > is then I could make of some library functions in reader macros off-target
> > and avoid installing libraries on the target.
>
> Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
> binary, right?

Correct.  I am working on a project that uses a MIPS processo embedded
in an SoC.  But I would like to take advantage of native, in
particular to allow me easy access to some hardware registers on the
SoC using mmap.  I need the 64-bit PicoLisp, but MIPS isn't one of the
architectures currently supported.  I started to look at how things
are done for ARM and intel, but don't have the time right now to
properly add MIPS.

So, I got the 64-bit PicoLisp compiled in emulator mode, after
cross-compiling sysdefs, capturing the output in a text file, and then
using that output in places where the output from sysdefs was read via
a pipe.

I had assumed (perhaps incorrectly) that, using emu, VM bytecode was
created on the fly and that it gets "executed" (I think this is a bit
how lua works - not LuaJIT).

> Note that normal PicoLisp doesn't compile anything. It "compiles" to 
> structures
> of pointers to pointers, where the compiler is called the "reader" in Lisp.
>
> As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
> source files to C structures, to be able to put them into ROM instead of
> precious RAM at runtime. However, these are still not byte codes, but the good
> old pointers to pointers :)

Believe or not, on this project, flash storage space is becoming a
problem.  Don't get me started on the decisions that led to this :).
That was my motivation for storing just the picolisp binary, and
finding some way to only "pay" for the storage space for the library
functions that I use.  I can collect them manually, I know, but I am
lazy and wanted to make the reader to my work for me.  I will look at
miniPicoLisp too.

I am completely open to the idea that I am being completely wrong
headed about something.

> ♪♫ Alex
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Jakob Eriksson]

Still, PicoLisp source, I mean the program you write in PicoLisp,
(not the C code) can become quite large depending on your application.

We mitigated this, not by storing the "compiled" structures of pointers
to pointers. (Although we could have - but that would have been
complicated to implemented I think, and I am not convinced it would have
been much smaller than the text source.)
Instead, we compressed the text files (picolisp source code) and
unpacked them upon on loading. Source code in general compresses very well.
In principle, I think you could create something like these Javascript
"minifiers" before putting it through compression. This hypothetical
PicoLisp minifier would for instance replace long variable names with
shorter ones.


https://github.com/mishoo/UglifyJS







On 2017-07-15 12:56, Alexander Burger wrote:
> Hi Matt,
> 
>> In order to minimize flash footprint, I was wondering if it would be
>> possible to compile code down to byte code?  The reason i wanted to do that
>> is then I could make of some library functions in reader macros off-target
>> and avoid installing libraries on the target.
> 
> Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
> binary, right?
> 
> Note that normal PicoLisp doesn't compile anything. It "compiles" to 
> structures
> of pointers to pointers, where the compiler is called the "reader" in Lisp.
> 
> As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
> source files to C structures, to be able to put them into ROM instead of
> precious RAM at runtime. However, these are still not byte codes, but the good
> old pointers to pointers :)
> 
> ♪♫ Alex
> 

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Alexander Burger]

Hi Matt,

> In order to minimize flash footprint, I was wondering if it would be
> possible to compile code down to byte code?  The reason i wanted to do that
> is then I could make of some library functions in reader macros off-target
> and avoid installing libraries on the target.

Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
binary, right?

Note that normal PicoLisp doesn't compile anything. It "compiles" to structures
of pointers to pointers, where the compiler is called the "reader" in Lisp.

As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
source files to C structures, to be able to put them into ROM instead of
precious RAM at runtime. However, these are still not byte codes, but the good
old pointers to pointers :)

♪♫ Alex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Compiling to bytecode

[Matt Wilbur]

Hi everyone,

In order to minimize flash footprint, I was wondering if it would be
possible to compile code down to byte code?  The reason i wanted to do that
is then I could make of some library functions in reader macros off-target
and avoid installing libraries on the target.

Matt