Re: Compiling to bytecode
Hi Matt, Nice plan you got! Maybe I can share you some of my experience implementing PIL64 to FPGA. But got some questions first: 1. Is it a 32bit or a 64bit MIPS? 2. Is it running an OS? I'm not sure but I think someone was attempting to port PicoLisp to an old SGI? And I'm sure that SGI was using MIPS, maybe that port might help you? BR, Geo -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Compiling to bytecode
On Sat, Jul 15, 2017 at 7:21 AM, Jakob Eriksson wrote: > Still, PicoLisp source, I mean the program you write in PicoLisp, > (not the C code) can become quite large depending on your application. > > We mitigated this, not by storing the "compiled" structures of pointers > to pointers. (Although we could have - but that would have been > complicated to implemented I think, and I am not convinced it would have > been much smaller than the text source.) > Instead, we compressed the text files (picolisp source code) and > unpacked them upon on loading. Source code in general compresses very well. > In principle, I think you could create something like these Javascript > "minifiers" before putting it through compression. This hypothetical > PicoLisp minifier would for instance replace long variable names with > shorter ones. This is another approach I that crossed my mind too, though I have only passing interaction with the JS world. I am assuming one could implement a load-like function that uses gzip and decompresses the stream in addition to reading. Matt > > > https://github.com/mishoo/UglifyJS > > > > > > > > On 2017-07-15 12:56, Alexander Burger wrote: >> Hi Matt, >> >>> In order to minimize flash footprint, I was wondering if it would be >>> possible to compile code down to byte code? The reason i wanted to do that >>> is then I could make of some library functions in reader macros off-target >>> and avoid installing libraries on the target. >> >> Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp >> binary, right? >> >> Note that normal PicoLisp doesn't compile anything. It "compiles" to >> structures >> of pointers to pointers, where the compiler is called the "reader" in Lisp. >> >> As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp >> source files to C structures, to be able to put them into ROM instead of >> precious RAM at runtime. However, these are still not byte codes, but the >> good >> old pointers to pointers :) >> >> ♪♫ Alex >> > > -- > UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Compiling to bytecode
On Sat, Jul 15, 2017 at 6:56 AM, Alexander Burger wrote: > > Hi Matt, > > > In order to minimize flash footprint, I was wondering if it would be > > possible to compile code down to byte code? The reason i wanted to do that > > is then I could make of some library functions in reader macros off-target > > and avoid installing libraries on the target. > > Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp > binary, right? Correct. I am working on a project that uses a MIPS processo embedded in an SoC. But I would like to take advantage of native, in particular to allow me easy access to some hardware registers on the SoC using mmap. I need the 64-bit PicoLisp, but MIPS isn't one of the architectures currently supported. I started to look at how things are done for ARM and intel, but don't have the time right now to properly add MIPS. So, I got the 64-bit PicoLisp compiled in emulator mode, after cross-compiling sysdefs, capturing the output in a text file, and then using that output in places where the output from sysdefs was read via a pipe. I had assumed (perhaps incorrectly) that, using emu, VM bytecode was created on the fly and that it gets "executed" (I think this is a bit how lua works - not LuaJIT). > Note that normal PicoLisp doesn't compile anything. It "compiles" to > structures > of pointers to pointers, where the compiler is called the "reader" in Lisp. > > As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp > source files to C structures, to be able to put them into ROM instead of > precious RAM at runtime. However, these are still not byte codes, but the good > old pointers to pointers :) Believe or not, on this project, flash storage space is becoming a problem. Don't get me started on the decisions that led to this :). That was my motivation for storing just the picolisp binary, and finding some way to only "pay" for the storage space for the library functions that I use. I can collect them manually, I know, but I am lazy and wanted to make the reader to my work for me. I will look at miniPicoLisp too. I am completely open to the idea that I am being completely wrong headed about something. > ♪♫ Alex > > -- > UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Floating Point numbers vs Fixpoint numbers
On Sat, Jul 15, 2017 at 04:16:47PM +0200, Jakob Eriksson wrote: > Oh, that was quite mundane - as most bugs are when found. :) > > Huge props to you for finding it so very fast. Thanks to Jimmie. His program produced the effect relatively often, so I could trace it down. Before I never observed it. It only occurs if garbage collection happens just when 'adduAE_A' is running. > >2460a2461 > >> push ZERO # Safe > >2489,2490c2490,2491 > >< ld E (L I) # Get product > >< ld (L I) A # Save halved argument > >call adduAE_A # Add for rounding > >--- > >> ld (L III) A # Save halved argument > >> ld E (L I) # Get product, keep in safe > >call adduAE_A # Add for rounding > > > > Register E was not saved before the call to 'adduAE_A', because the stack > > entry (L I) was overwritten with register A. Now A is saved in a separate > > entry (L III). > > > > ♪♫ Alex > > > > -- > UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Floating Point numbers vs Fixpoint numbers
Oh, that was quite mundane - as most bugs are when found. :) Huge props to you for finding it so very fast. On 2017-07-15 14:45, Alexander Burger wrote: > On Sat, Jul 15, 2017 at 02:02:00PM +0200, Jakob Eriksson wrote: >> What was it? > > The diff is: > >2460a2461 >> push ZERO # Safe >2489,2490c2490,2491 >< ld E (L I) # Get product >< ld (L I) A # Save halved argument >call adduAE_A # Add for rounding >--- >> ld (L III) A # Save halved argument >> ld E (L I) # Get product, keep in safe >call adduAE_A # Add for rounding > > Register E was not saved before the call to 'adduAE_A', because the stack > entry (L I) was overwritten with register A. Now A is saved in a separate > entry (L III). > > ♪♫ Alex > -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Subscribe
Re: Floating Point numbers vs Fixpoint numbers
On Sat, Jul 15, 2017 at 02:02:00PM +0200, Jakob Eriksson wrote: > What was it? The diff is: 2460a2461 > push ZERO # Safe 2489,2490c2490,2491 < ld E (L I) # Get product < ld (L I) A # Save halved argument call adduAE_A # Add for rounding --- > ld (L III) A # Save halved argument > ld E (L I) # Get product, keep in safe call adduAE_A # Add for rounding Register E was not saved before the call to 'adduAE_A', because the stack entry (L I) was overwritten with register A. Now A is saved in a separate entry (L III). ♪♫ Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Floating Point numbers vs Fixpoint numbers
What was it? On 2017-07-15 13:51, Alexander Burger wrote: > On Sat, Jul 15, 2017 at 12:52:52PM +0200, Alexander Burger wrote: >> I cound not locate the exact location of the bug yet. It is a Heisenbug, >> because >> it only occurs if the garbage collector runs in the right moment. I know this >> ... >> I will dig into it now, to find and fix it, and release a new PicoLisp >> version. > > Done! I found the culprit in the '*/' function! > > A new version is available as always in > >https://software-lab.de/picoLisp.tgz > > Thanks again! > ♪♫ Alex > -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Floating Point numbers vs Fixpoint numbers
On Sat, Jul 15, 2017 at 12:52:52PM +0200, Alexander Burger wrote: > I cound not locate the exact location of the bug yet. It is a Heisenbug, > because > it only occurs if the garbage collector runs in the right moment. I know this > ... > I will dig into it now, to find and fix it, and release a new PicoLisp > version. Done! I found the culprit in the '*/' function! A new version is available as always in https://software-lab.de/picoLisp.tgz Thanks again! ♪♫ Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Compiling to bytecode
Still, PicoLisp source, I mean the program you write in PicoLisp, (not the C code) can become quite large depending on your application. We mitigated this, not by storing the "compiled" structures of pointers to pointers. (Although we could have - but that would have been complicated to implemented I think, and I am not convinced it would have been much smaller than the text source.) Instead, we compressed the text files (picolisp source code) and unpacked them upon on loading. Source code in general compresses very well. In principle, I think you could create something like these Javascript "minifiers" before putting it through compression. This hypothetical PicoLisp minifier would for instance replace long variable names with shorter ones. https://github.com/mishoo/UglifyJS On 2017-07-15 12:56, Alexander Burger wrote: > Hi Matt, > >> In order to minimize flash footprint, I was wondering if it would be >> possible to compile code down to byte code? The reason i wanted to do that >> is then I could make of some library functions in reader macros off-target >> and avoid installing libraries on the target. > > Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp > binary, right? > > Note that normal PicoLisp doesn't compile anything. It "compiles" to > structures > of pointers to pointers, where the compiler is called the "reader" in Lisp. > > As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp > source files to C structures, to be able to put them into ROM instead of > precious RAM at runtime. However, these are still not byte codes, but the good > old pointers to pointers :) > > ♪♫ Alex > -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Compiling to bytecode
Hi Matt, > In order to minimize flash footprint, I was wondering if it would be > possible to compile code down to byte code? The reason i wanted to do that > is then I could make of some library functions in reader macros off-target > and avoid installing libraries on the target. Can you explain what you mean? Compile *what* to byte code? Not the PicoLisp binary, right? Note that normal PicoLisp doesn't compile anything. It "compiles" to structures of pointers to pointers, where the compiler is called the "reader" in Lisp. As you talk of libraries, you should look at miniPicoLisp. It "compiles" Lisp source files to C structures, to be able to put them into ROM instead of precious RAM at runtime. However, these are still not byte codes, but the good old pointers to pointers :) ♪♫ Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Floating Point numbers vs Fixpoint numbers
Hi Jimmie, thanks for your long feedback and sharing your experiences! > I have spent a fair amount of the last 4 days playing with PicoLisp. I Yess. From your code I see that you master PicoLisp already quite well! I suspect that you gave up just a little bit too early. > : (scl 12) > -> 12 > : (*Scl) > Segmentation fault (core dumped) Haha, right. Calling (*Scl) is a prefect way to segfault. Think about what it means: It calls a *function* starting in memory at address 12! HOWEVER! The segfaults you observed in your program are NOT your fault! You indeed found a bug in the bignum functions of PicoLisp, causing a crash upon a certain combination of numeric arguments. Strange that this was not detected earlier, I used a lot of test routines with random numbers. I cound not locate the exact location of the bug yet. It is a Heisenbug, because it only occurs if the garbage collector runs in the right moment. I know this because it all runs well if you call 'gc' *before* the test, e.g. (gc 700) (doit) I will dig into it now, to find and fix it, and release a new PicoLisp version. Thanks for finding it! As you spent already so much time on it, I feel that I should try it myself. I rewrote parts of it to be more PicoLisp-like, as you violate some of the PicoLisp programming rules and styles. I won't comment on the details, but paste my own version below. The new version is about 150 times faster. The main reason is that you used 'nth' to index into long lists again and again. Here on my notebook it finishes the 100 reps in one and a half minutes: $ time ./pil x.l + *Scl: 12 createlist "loop1" 87.144 sec loop2 nsum: 11257.349310772499 navg: 0.390880184402 4.093 sec 91.250 sec nsum: 11228.585386471924 navg: 0.389881437030 real1m31.628s user1m31.308s sys 0m0.400s > A successful 100 rep run should give an > nsum: 11322.307098752284and > navg: 0.393135663151121within acceptable floating point differences. Note that I get slightly different results, because I use '*/' instead of '/' in the 'average' function. I believe that this is more correct, as '*/' rounds the result. Now let me start the bug hunt. ♪♫ Alex PS: Here is my code: (scl 12) (de average (L) (*/ (sum prog L) (length L)) ) (de normalize (N) (let NN (if (=0 N) 0.000123456789 N) (while (<= NN 0.0001) (setq NN (* 10 NN)) ) (while (>= NN 1.0) (setq NN (*/ NN 10)) ) NN ) ) (de createlist (Lsize) (make (let I 0 (do Lsize (link (normalize (inc 'I 1.0))) ) ) ) ) (de loop1calc (I J N) (let V (*/ N (+ I N) (- J N) 0.1234567 `(* 1.0 1.0 1.0)) (normalize (*/ V V V `(* 1.0 1.0)) ) ) ) (de loop2calc (I J N) (normalize (*/ N (+ J 1.0) (+ N N N) 0.1234567 `(* 1.0 1.0 1.0) ) ) ) (de loopN (Fun Lst Reps) (let I 1.0 (do Reps (let J 0 (map '((L) (set L (Fun I (inc 'J 1.0) (car L)) ) ) Lst ) ) (inc 'I 1.0) ) ) (setq *Nsum (sum prog Lst) *Navg (average Lst) ) ) (de loop1 (Lst Reps) (loopN loop1calc Lst Reps) ) (de loop2 (Lst Reps) (loopN loop2calc Lst Reps) ) (de doit () (prinl "*Scl: " *Scl) (bench (let Reps 100 (prinl "createlist") (let L (createlist (* 60 24 5 4)) (println "loop1") (bench (loop1 L Reps)) (prinl "loop2") (prinl "nsum: " (format *Nsum *Scl)) (prinl "navg: " (format *Navg *Scl)) (bench (loop2 L Reps)) ) ) ) (prinl "nsum: " (format *Nsum *Scl)) (prinl "navg: " (format *Navg *Scl)) ) (gc 700) (doit) (bye) -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe