Re: Compiling to bytecode

[George Orais]

Hi Matt,

Nice plan you got!

Maybe I can share you some of my experience implementing PIL64 to FPGA.

But got some questions first:
1. Is it a 32bit or a 64bit MIPS?
2. Is it running an OS?

I'm not sure but I think someone was attempting to port PicoLisp to an old SGI? 
And I'm sure that SGI was using MIPS, maybe that port might help you?


BR,
Geo

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Matt Wilbur]

On Sat, Jul 15, 2017 at 7:21 AM, Jakob Eriksson  wrote:
> Still, PicoLisp source, I mean the program you write in PicoLisp,
> (not the C code) can become quite large depending on your application.
>
> We mitigated this, not by storing the "compiled" structures of pointers
> to pointers. (Although we could have - but that would have been
> complicated to implemented I think, and I am not convinced it would have
> been much smaller than the text source.)
> Instead, we compressed the text files (picolisp source code) and
> unpacked them upon on loading. Source code in general compresses very well.
> In principle, I think you could create something like these Javascript
> "minifiers" before putting it through compression. This hypothetical
> PicoLisp minifier would for instance replace long variable names with
> shorter ones.

This is another approach I that crossed my mind too, though I have
only passing interaction with the JS world.
I am assuming one could implement a load-like function that uses gzip
and decompresses the stream in addition to reading.

Matt
>
>
> https://github.com/mishoo/UglifyJS
>
>
>
>
>
>
>
> On 2017-07-15 12:56, Alexander Burger wrote:
>> Hi Matt,
>>
>>> In order to minimize flash footprint, I was wondering if it would be
>>> possible to compile code down to byte code?  The reason i wanted to do that
>>> is then I could make of some library functions in reader macros off-target
>>> and avoid installing libraries on the target.
>>
>> Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
>> binary, right?
>>
>> Note that normal PicoLisp doesn't compile anything. It "compiles" to 
>> structures
>> of pointers to pointers, where the compiler is called the "reader" in Lisp.
>>
>> As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
>> source files to C structures, to be able to put them into ROM instead of
>> precious RAM at runtime. However, these are still not byte codes, but the 
>> good
>> old pointers to pointers :)
>>
>> ♪♫ Alex
>>
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Matt Wilbur]

On Sat, Jul 15, 2017 at 6:56 AM, Alexander Burger  wrote:
>
> Hi Matt,
>
> > In order to minimize flash footprint, I was wondering if it would be
> > possible to compile code down to byte code?  The reason i wanted to do that
> > is then I could make of some library functions in reader macros off-target
> > and avoid installing libraries on the target.
>
> Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
> binary, right?

Correct.  I am working on a project that uses a MIPS processo embedded
in an SoC.  But I would like to take advantage of native, in
particular to allow me easy access to some hardware registers on the
SoC using mmap.  I need the 64-bit PicoLisp, but MIPS isn't one of the
architectures currently supported.  I started to look at how things
are done for ARM and intel, but don't have the time right now to
properly add MIPS.

So, I got the 64-bit PicoLisp compiled in emulator mode, after
cross-compiling sysdefs, capturing the output in a text file, and then
using that output in places where the output from sysdefs was read via
a pipe.

I had assumed (perhaps incorrectly) that, using emu, VM bytecode was
created on the fly and that it gets "executed" (I think this is a bit
how lua works - not LuaJIT).

> Note that normal PicoLisp doesn't compile anything. It "compiles" to 
> structures
> of pointers to pointers, where the compiler is called the "reader" in Lisp.
>
> As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
> source files to C structures, to be able to put them into ROM instead of
> precious RAM at runtime. However, these are still not byte codes, but the good
> old pointers to pointers :)

Believe or not, on this project, flash storage space is becoming a
problem.  Don't get me started on the decisions that led to this :).
That was my motivation for storing just the picolisp binary, and
finding some way to only "pay" for the storage space for the library
functions that I use.  I can collect them manually, I know, but I am
lazy and wanted to make the reader to my work for me.  I will look at
miniPicoLisp too.

I am completely open to the idea that I am being completely wrong
headed about something.

> ♪♫ Alex
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Floating Point numbers vs Fixpoint numbers

[Jakob Eriksson]

Oh, that was quite mundane - as most bugs are when found. :)

Huge props to you for finding it so very fast.


On 2017-07-15 14:45, Alexander Burger wrote:
> On Sat, Jul 15, 2017 at 02:02:00PM +0200, Jakob Eriksson wrote:
>> What was it?
> 
> The diff is:
> 
>2460a2461
>>   push ZERO  #  Safe
>2489,2490c2490,2491
><   ld E (L I)  # Get product
><   ld (L I) A  # Save halved argument
>call adduAE_A  # Add for rounding
>---
>>   ld (L III) A  # Save halved argument
>>   ld E (L I)  # Get product, keep in safe
>call adduAE_A  # Add for rounding
> 
> Register E was not saved before the call to 'adduAE_A', because the stack
> entry (L I) was overwritten with register A. Now A is saved in a separate
> entry (L III).
> 
> ♪♫ Alex
> 

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Subscribe

[Alfonso Villén]

Re: Floating Point numbers vs Fixpoint numbers

[Alexander Burger]

On Sat, Jul 15, 2017 at 02:02:00PM +0200, Jakob Eriksson wrote:
> What was it?

The diff is:

   2460a2461
   >   push ZERO  #  Safe
   2489,2490c2490,2491
   <   ld E (L I)  # Get product
   <   ld (L I) A  # Save halved argument
   call adduAE_A  # Add for rounding
   ---
   >   ld (L III) A  # Save halved argument
   >   ld E (L I)  # Get product, keep in safe
   call adduAE_A  # Add for rounding

Register E was not saved before the call to 'adduAE_A', because the stack
entry (L I) was overwritten with register A. Now A is saved in a separate
entry (L III).

♪♫ Alex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Floating Point numbers vs Fixpoint numbers

[Jakob Eriksson]

What was it?


On 2017-07-15 13:51, Alexander Burger wrote:
> On Sat, Jul 15, 2017 at 12:52:52PM +0200, Alexander Burger wrote:
>> I cound not locate the exact location of the bug yet. It is a Heisenbug, 
>> because
>> it only occurs if the garbage collector runs in the right moment. I know this
>> ...
>> I will dig into it now, to find and fix it, and release a new PicoLisp 
>> version.
> 
> Done! I found the culprit in the '*/' function!
> 
> A new version is available as always in
> 
>https://software-lab.de/picoLisp.tgz
> 
> Thanks again!
> ♪♫ Alex
> 

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Floating Point numbers vs Fixpoint numbers

[Alexander Burger]

On Sat, Jul 15, 2017 at 12:52:52PM +0200, Alexander Burger wrote:
> I cound not locate the exact location of the bug yet. It is a Heisenbug, 
> because
> it only occurs if the garbage collector runs in the right moment. I know this
> ...
> I will dig into it now, to find and fix it, and release a new PicoLisp 
> version.

Done! I found the culprit in the '*/' function!

A new version is available as always in

   https://software-lab.de/picoLisp.tgz

Thanks again!
♪♫ Alex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Jakob Eriksson]

Still, PicoLisp source, I mean the program you write in PicoLisp,
(not the C code) can become quite large depending on your application.

We mitigated this, not by storing the "compiled" structures of pointers
to pointers. (Although we could have - but that would have been
complicated to implemented I think, and I am not convinced it would have
been much smaller than the text source.)
Instead, we compressed the text files (picolisp source code) and
unpacked them upon on loading. Source code in general compresses very well.
In principle, I think you could create something like these Javascript
"minifiers" before putting it through compression. This hypothetical
PicoLisp minifier would for instance replace long variable names with
shorter ones.


https://github.com/mishoo/UglifyJS







On 2017-07-15 12:56, Alexander Burger wrote:
> Hi Matt,
> 
>> In order to minimize flash footprint, I was wondering if it would be
>> possible to compile code down to byte code?  The reason i wanted to do that
>> is then I could make of some library functions in reader macros off-target
>> and avoid installing libraries on the target.
> 
> Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
> binary, right?
> 
> Note that normal PicoLisp doesn't compile anything. It "compiles" to 
> structures
> of pointers to pointers, where the compiler is called the "reader" in Lisp.
> 
> As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
> source files to C structures, to be able to put them into ROM instead of
> precious RAM at runtime. However, these are still not byte codes, but the good
> old pointers to pointers :)
> 
> ♪♫ Alex
> 

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Compiling to bytecode

[Alexander Burger]

Hi Matt,

> In order to minimize flash footprint, I was wondering if it would be
> possible to compile code down to byte code?  The reason i wanted to do that
> is then I could make of some library functions in reader macros off-target
> and avoid installing libraries on the target.

Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp
binary, right?

Note that normal PicoLisp doesn't compile anything. It "compiles" to structures
of pointers to pointers, where the compiler is called the "reader" in Lisp.

As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp
source files to C structures, to be able to put them into ROM instead of
precious RAM at runtime. However, these are still not byte codes, but the good
old pointers to pointers :)

♪♫ Alex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Floating Point numbers vs Fixpoint numbers

[Alexander Burger]

Hi Jimmie,

thanks for your long feedback and sharing your experiences!

> I have spent a fair amount of the last 4 days playing with PicoLisp. I

Yess. From your code I see that you master PicoLisp already quite well! I
suspect that you gave up just a little bit too early.


> : (scl 12)
> -> 12
> : (*Scl)
> Segmentation fault (core dumped)

Haha, right. Calling (*Scl) is a prefect way to segfault. Think about what it
means: It calls a *function* starting in memory at address 12!


HOWEVER! The segfaults you observed in your program are NOT your fault! You
indeed found a bug in the bignum functions of PicoLisp, causing a crash upon a
certain combination of numeric arguments.

Strange that this was not detected earlier, I used a lot of test routines with
random numbers.

I cound not locate the exact location of the bug yet. It is a Heisenbug, because
it only occurs if the garbage collector runs in the right moment. I know this
because it all runs well if you call 'gc' *before* the test, e.g.

   (gc 700)
   (doit)

I will dig into it now, to find and fix it, and release a new PicoLisp version.
Thanks for finding it!


As you spent already so much time on it, I feel that I should try it myself.

I rewrote parts of it to be more PicoLisp-like, as you violate some of the
PicoLisp programming rules and styles. I won't comment on the details, but paste
my own version below.

The new version is about 150 times faster. The main reason is that you used
'nth' to index into long lists again and again. Here on my notebook it
finishes the 100 reps in one and a half minutes:

   $ time ./pil x.l +
   *Scl: 12
   createlist
   "loop1"
   87.144 sec
   loop2
   nsum: 11257.349310772499
   navg: 0.390880184402
   4.093 sec
   91.250 sec
   nsum: 11228.585386471924
   navg: 0.389881437030

   real1m31.628s
   user1m31.308s
   sys 0m0.400s


> A successful 100 rep run should give an
> nsum: 11322.307098752284and
> navg: 0.393135663151121within acceptable floating point differences.

Note that I get slightly different results, because I use '*/' instead of '/' in
the 'average' function. I believe that this is more correct, as '*/' rounds the
result.


Now let me start the bug hunt.

♪♫ Alex


PS: Here is my code:

(scl 12)

(de average (L)
   (*/ (sum prog L) (length L)) )

(de normalize (N)
   (let NN (if (=0 N) 0.000123456789 N)
  (while (<= NN 0.0001)
 (setq NN (* 10 NN)) )
  (while (>= NN 1.0)
 (setq NN (*/ NN 10)) )
  NN ) )

(de createlist (Lsize)
   (make
  (let I 0
 (do Lsize
(link (normalize (inc 'I 1.0))) ) ) ) )

(de loop1calc (I J N)
   (let V (*/ N (+ I N) (- J N) 0.1234567 `(* 1.0 1.0 1.0))
  (normalize (*/ V V V `(* 1.0 1.0)) ) ) )

(de loop2calc (I J N)
   (normalize
  (*/ N
 (+ J 1.0)
 (+ N N N)
 0.1234567
 `(* 1.0 1.0 1.0) ) ) )

(de loopN (Fun Lst Reps)
   (let I 1.0
  (do Reps
 (let J 0
(map
   '((L)
  (set L
 (Fun I (inc 'J 1.0) (car L)) ) )
   Lst ) )
 (inc 'I 1.0) ) )
   (setq
  *Nsum (sum prog Lst)
  *Navg (average Lst) ) )

(de loop1 (Lst Reps)
   (loopN loop1calc Lst Reps) )

(de loop2 (Lst Reps)
   (loopN loop2calc Lst Reps) )

(de doit ()
   (prinl "*Scl: " *Scl)
   (bench
  (let Reps 100
 (prinl "createlist")
 (let L (createlist (* 60 24 5 4))
(println "loop1")
(bench (loop1 L Reps))
(prinl "loop2")
(prinl "nsum: " (format *Nsum *Scl))
(prinl "navg: " (format *Navg *Scl))
(bench (loop2 L Reps)) ) ) )
   (prinl "nsum: " (format *Nsum *Scl))
   (prinl "navg: " (format *Navg *Scl)) )

(gc 700)
(doit)
(bye)

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe