Re: picoLisp 19.12: variable length array in structure fixes

[C K Kashyap]

Super ...Congratulations Andras!

I just gave it a spin and it just worked. I generated 64bit and 32bit
executables (after installing libffi-dev and libffi-dev:i386) and tried
(call "ls") :)
I am particularly happy since I will be able to reference C code for the
FFI implementation that I could potentially use in miniPicoLisp.

Regards,
Kashyap


On Fri, May 22, 2020 at 10:05 AM Alexander Burger 
wrote:

> Hi Andras,
>
> > The code at https://github.com/pahihu/picoLisp <
> https://github.com/pahihu/picoLisp> fulfils Mike’s super
> > goal - all pil64 specific tests passed as well. It has namespaces,
> > native C calls and coroutines and the documented features of
> > pil64 - except the external symbol and database format.
>
> Wow! Big achievement, thanks a lot!!
>
> ☺/ A!ex
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
>

Re: picoLisp 19.12: variable length array in structure fixes

[Alexander Burger]

Hi Andras,

> The code at https://github.com/pahihu/picoLisp 
>  fulfils Mike’s super
> goal - all pil64 specific tests passed as well. It has namespaces,
> native C calls and coroutines and the documented features of
> pil64 - except the external symbol and database format.

Wow! Big achievement, thanks a lot!!

☺/ A!ex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Andras Pahi]

Hi all,

The code at https://github.com/pahihu/picoLisp 
 fulfils Mike’s super
goal - all pil64 specific tests passed as well. It has namespaces,
native C calls and coroutines and the documented features of
pil64 - except the external symbol and database format.

Andras

> On 2020. Apr 19., at 13:11, Mike  wrote:
> !!!
> Super goal - huge.l should pass all code with your patch set.
> 
> 
> p.s. I dont have enough energy right now to debug all this.  
> 
> (mike)

Re: picoLisp 19.12: variable length array in structure fixes

[Guido Stepken]

Interesting. You might have also run into no "excution bits" on Intel
hardware:

https://releases.llvm.org/8.0.1/docs/SpeculativeLoadHardening.html

https://en.wikipedia.org/wiki/NX_bit

In Lisp, "code is data, data is code". There simply is no separation the
like - "code here, data there". Typical Lisp JIT emits machine code, where
needed.

This is a common and well known problem. Unsure, if you can disable that
behavior in Apple BIOS. And even then you must recompile with options.

US hardware and software stacks increasingly become nebulous.

Have fun, Guido Stepken

Am Dienstag, 21. April 2020 schrieb Andras Pahi :
> Hi,
>
> Maybe not related to this one, but on Mac OS X the heap size is limited
> to 65532KB. On startup picolisp fails to set the unlimited stack size,
> And use the actual ulimit so on the first run I’ve got a SIGSEGV running
> code2015.l
>
> Andras
>
>
>> On 2020. Apr 21., at 10:38, Mike  wrote:
>>
>> hi all,
>>
>>> If you are interested I have patched the 19.12 32bit sources to compile
without GCC.
>>> I have attached the changed files: pico.h, main.c, apply.c and flow.c
>>>
>>> Since clang does not support variable length array in structures I
allocate the bindFrame
>>> with alloca() and provided a macro in pico.h to ease this: allocFrame()

Re: picoLisp 19.12: variable length array in structure fixes

[Mike]

> Maybe not related to this one, but on Mac OS X the heap size is limited
> to 65532KB. On startup picolisp fails to set the unlimited stack size,
> And use the actual ulimit so on the first run I’ve got a SIGSEGV running 
> code2015.l
> 

Indeed! I always forget this requirement on Openbsd.
$ ulimit -s 32000
and it passed malloc guards.
Thanks.

(mike)

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Andras Pahi]

Hi,

Maybe not related to this one, but on Mac OS X the heap size is limited
to 65532KB. On startup picolisp fails to set the unlimited stack size,
And use the actual ulimit so on the first run I’ve got a SIGSEGV running 
code2015.l

Andras


> On 2020. Apr 21., at 10:38, Mike  wrote:
> 
> hi all,
> 
>> If you are interested I have patched the 19.12 32bit sources to compile 
>> without GCC.
>> I have attached the changed files: pico.h, main.c, apply.c and flow.c
>> 
>> Since clang does not support variable length array in structures I allocate 
>> the bindFrame
>> with alloca() and provided a macro in pico.h to ease this: allocFrame().
>> 
>> I know that the 32bit version is not the mainstream version, but feel free to
>> abuse the patches.
> 
> FYI
> 
> Breaking news from secret testing laboratory:
> pil32-original and pil32-VLA killed by malloc guard in OpenBSD 6.6+syspatch:
> $ sysctl -w vm.malloc_conf="SC"
> $ pil code2015.l +
> Segmentation fault (core dumped)
> $ dmesg | tail -1
> [picolisp]35577/479639 sp=cefce000 inside cefcf000-cf7ce000: not MAP_STACK
> 
> p.s. BTW, ASan from gcc and clang - ok
> p.s.s. Several years ago (6.2-6.4 ?) it worked.
> 
> (mike)
> 
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe


--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Mike]

hi all,

> If you are interested I have patched the 19.12 32bit sources to compile 
> without GCC.
> I have attached the changed files: pico.h, main.c, apply.c and flow.c
> 
> Since clang does not support variable length array in structures I allocate 
> the bindFrame
> with alloca() and provided a macro in pico.h to ease this: allocFrame().
> 
> I know that the 32bit version is not the mainstream version, but feel free to
> abuse the patches.

FYI

Breaking news from secret testing laboratory:
pil32-original and pil32-VLA killed by malloc guard in OpenBSD 6.6+syspatch:
$ sysctl -w vm.malloc_conf="SC"
$ pil code2015.l +
Segmentation fault (core dumped)
$ dmesg | tail -1
[picolisp]35577/479639 sp=cefce000 inside cefcf000-cf7ce000: not MAP_STACK

p.s. BTW, ASan from gcc and clang - ok
p.s.s. Several years ago (6.2-6.4 ?) it worked.

(mike)

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Guido Stepken]

Linux, here Debian GNU/Linux, allows parallel installation of 32/64 bit
libraries ...

https://wiki.debian.org/Multiarch/HOWTO

Sometimes "apt-get install ia32-libs" helps.

Have fun!

Am Montag, 20. April 2020 schrieb Mike :
> April 20, 2020 11:01 AM, "Andras Pahi"  wrote:
>
>> Hi Mike,
>>
>> pil32, x64 means you’ve built the contents of the src/ dir in x64 mode ?
>>
>
> pil32, x64 - means gcc and voidlinux-x64 compiled-supports multilib.
> I use pil32 and pil64 on the same machine, I always in pil64, but can
switch fast.
>
> (mike)
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subjectUnsubscribe
>

Re: picoLisp 19.12: variable length array in structure fixes

[Mike]

April 20, 2020 11:01 AM, "Andras Pahi"  wrote:

> Hi Mike,
> 
> pil32, x64 means you’ve built the contents of the src/ dir in x64 mode ?
> 

pil32, x64 - means gcc and voidlinux-x64 compiled-supports multilib.
I use pil32 and pil64 on the same machine, I always in pil64, but can switch 
fast.

(mike)

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Andras Pahi]

Hi Mike,

pil32, x64 means you’ve built the contents of the src/ dir in x64 mode ?

Thanks,
Andras

> There are more numbers for the only task "advent2015":
> pil64, x64 - 1 min
> pil32, x64 - 9 min
> pil32, 686 - 9 min
> pil32, 686, -O3 - 7 min
> VLA patch, 686 - 10 min
> 
> (mike)
> 
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe


--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Mike]

> 
> You don’t need energy, you need some patience.
> 

Hard get patience when code running 9x slower :)
Its awesome your patch is working and correct, congratulations!

There are more numbers for the only task "advent2015":
pil64, x64 - 1 min
pil32, x64 - 9 min
pil32, 686 - 9 min
pil32, 686, -O3 - 7 min
VLA patch, 686 - 10 min

p.s. I will create patch and will store in tankf33der repo.
  
(mike)

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Andras Pahi]

Hi Mike,

You don’t need energy, you need some patience.

Andras

===> time pil huge.l -bye
OK-UTF8
OK-Montgomery
OK-Leibniz
OK-math
OK-forth
OK-parse2list
OK-mapreduce
OK-pow
OK-destr
OK-test-car
OK-bitwise
# reduce redefined
# pow redefined
OK-4clojure
# pow redefined
OK-AoC15
OK-SimplyScheme
# lcm redefined
# evenp redefined
# second redefined
# mystery redefined
# add1 redefined
# numbers redefined
# half redefined
OK-Touretzky
# prime? redefined
OK-Menshikov
# pow redefined
# exp redefined
# log redefined
# sin redefined
# cos redefined
# tan redefined
# asin redefined
# acos redefined
# atan redefined
# atan2 redefined
# create redefined
# inrow redefined
# f3-1 redefined
# f3-2 redefined
# f3-3 redefined
# f4-2 redefined
# f4-3 redefined
# f8-22 redefined
OK-Gorlansky
OK-Stepanoff
SIPHASH-OK
# mod32+ redefined
# endian redefined
Hash-ALL-OK
OK-RNG
OK-HKDF
OK-test-pbkdf2
# pow redefined
# exp redefined
# log redefined
# sin redefined
# cos redefined
# tan redefined
# asin redefined
# acos redefined
# atan redefined
# atan2 redefined
OK-HMAC
OK-test-MD5
OK-test-MD2
OK-test-aes
# totwo redefined
# fromtwo redefined
# tilde redefined
OK-test-KECCAK
OK-test-SHA512
OK-test-SHA256
OK-test-RIPEMD160
# mod32+ redefined
# hex2L redefined
OK-test-Blowfish
OK-test-bcrypt
# mod32+ redefined
# mod32* redefined
# hex2L redefined
OK-test-twofish
# endian redefined
OK-test-threefish
# hex2L redefined
OK-test-RC2
# mod32* redefined
# hex2L redefined
OK-test-rc5
OK-test-rc6
# f redefined
# num64 redefined
# create redefined
OK-test-camellia
# expand redefined
OK-test-idea
OK-test-MD4
OK-test-SHA1
# num64 redefined
# sbox redefined
OK-test-Tiger
# wsbox redefined
OK-test-Whirlpool
OK-test-JH256
# num32 redefined
OK-test-anubis
# rol redefined
OK-test-scrypt
# endian redefined
OK-test-blake2s
# endian redefined
OK-test-blake2b
# i redefined
OK-test-Kangaroo12
crypto-ALL-OK
OK-All

real13m46.284s
user13m58.501s
sys 0m15.341s


> On 2020. Apr 19., at 13:11, Mike  wrote:
> 
> hi all,
> 
>> If you are interested I have patched the 19.12 32bit sources to compile 
>> without GCC.
>> I have attached the changed files: pico.h, main.c, apply.c and flow.c
> 
> My testing status for https://github.com/picolisp/picolisp
> 
> 1. pil @lib/test.l +
> clang  - ok
> clang+asan - ok
> tcc-git- ok
> 
> 
> 2. huge.l - failed (you must take care)
> $ git clone https://git.envs.net/mpech/tankf33der.git
> $ cd tankf33der
> $ pil huge.l + 
> OK-UTF8
> OK-Montgomery
> OK-Leibniz
> OK-math
> OK-forth
> OK-parse2list
> OK-mapreduce
> OK-pow
> OK-destr
> OK-test-car
> OK-bitwise
> # reduce redefined
> # pow redefined
> OK-4clojure
> # pow redefined
> // hangs in advent2015/code2015.l
> // is it hangs or very-very slow?
> // the same in crypto/test.l
> 
> !!!
> Super goal - huge.l should pass all code with your patch set.
> 
> 
> p.s. I dont have enough energy right now to debug all this.  
> 
> (mike)
> 
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe


--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[George-Phillip Orais]

Hi tankf33der,

Thanks for sharing your huge.l test, I tried it on my pil64 on WSL and all
passed!

20:27 $ pil huge.l +
OK-UTF8
OK-Montgomery
OK-Leibniz
OK-math
OK-forth
OK-parse2list
OK-mapreduce
OK-pow
OK-inc-db
# worker redefined
# +Inc c redefined
OK-inc-db-v2
OK-destr
OK-test-car
OK-bitwise
# reduce redefined
OK-Cartesian
# reduce redefined
# pow redefined
# seq redefined
OK-4clojure
# pow redefined
OK-AoC15
OK-AoC16
# f3-2 redefined
# f4-2 redefined
OK-AoC17
# a10 redefined
OK-AoC18
# c5 redefined
# c9 redefined
# c11 redefined
# c13 redefined
OK-AoC19
OK-SimplyScheme
# lcm redefined
# evenp redefined
# second redefined
# mystery redefined
# add1 redefined
# numbers redefined
# half redefined
OK-Touretzky
# prime? redefined
OK-Menshikov
# pow redefined
# exp redefined
# log redefined
# sin redefined
# cos redefined
# tan redefined
# asin redefined
# acos redefined
# atan redefined
# atan2 redefined
# f1-1 redefined
# f1-2 redefined
# create redefined
# inrow redefined
# f2-1 redefined
# move redefined
# f3-1 redefined
# f3-2 redefined
# f3-3 redefined
# f4-1 redefined
# f4-2 redefined
# f4-3 redefined
# product redefined
# f5-1 redefined
# f5-2 redefined
# f6-1 redefined
# f6-2 redefined
# f7-1 redefined
# f7-2 redefined
# f8-22 redefined
# f10-1 redefined
# f10-2 redefined
# f14-1 redefined
# f14-2 redefined
# f15-1 redefined
# f16-2 redefined
# f17-1 redefined
# f17-2 redefined
# f18-1 redefined
# f18-2 redefined
OK-Gorlansky
OK-Stepanoff
SIPHASH-OK
# mod32+ redefined
# endian redefined
Hash-ALL-OK
OK-RNG
OK-HKDF
OK-test-pbkdf2
# pow redefined
# exp redefined
# log redefined
# sin redefined
# cos redefined
# tan redefined
# asin redefined
# acos redefined
# atan redefined
# atan2 redefined
# T1 redefined
OK-HMAC
OK-test-MD5
OK-test-MD2
OK-test-aes
# totwo redefined
# fromtwo redefined
# tilde redefined
OK-test-KECCAK
OK-test-SHA512
OK-test-SHA256
OK-test-RIPEMD160
# mod32+ redefined
# hex2L redefined
OK-test-Blowfish
OK-test-bcrypt
# mod32+ redefined
# mod32* redefined
# hex2L redefined
OK-test-twofish
# endian redefined
OK-test-threefish
# hex2L redefined
OK-test-RC2
# mod32* redefined
# hex2L redefined
OK-test-rc5
OK-test-rc6
# f redefined
# num64 redefined
# create redefined
OK-test-camellia
# expand redefined
OK-test-idea
OK-test-MD4
OK-test-SHA1
# num64 redefined
# sbox redefined
OK-test-Tiger
# wsbox redefined
OK-test-Whirlpool
OK-test-JH256
# num32 redefined
OK-test-anubis
# rol redefined
OK-test-scrypt
# endian redefined
OK-test-blake2s
# endian redefined
OK-test-blake2b
# i redefined
OK-test-Kangaroo12
crypto-ALL-OK
OK-All

Cool! Thanks!


BR,
Geo

On Sun, Apr 19, 2020 at 8:17 PM Mike  wrote:

> hi all,
>
> > If you are interested I have patched the 19.12 32bit sources to compile
> without GCC.
> > I have attached the changed files: pico.h, main.c, apply.c and flow.c
>
> My testing status for https://github.com/picolisp/picolisp
>
> 1. pil @lib/test.l +
> clang  - ok
> clang+asan - ok
> tcc-git- ok
>
>
> 2. huge.l - failed (you must take care)
> $ git clone https://git.envs.net/mpech/tankf33der.git
> $ cd tankf33der
> $ pil huge.l +
> OK-UTF8
> OK-Montgomery
> OK-Leibniz
> OK-math
> OK-forth
> OK-parse2list
> OK-mapreduce
> OK-pow
> OK-destr
> OK-test-car
> OK-bitwise
> # reduce redefined
> # pow redefined
> OK-4clojure
> # pow redefined
> // hangs in advent2015/code2015.l
> // is it hangs or very-very slow?
> // the same in crypto/test.l
>
> !!!
> Super goal - huge.l should pass all code with your patch set.
>
>
> p.s. I dont have enough energy right now to debug all this.
>
> (mike)
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subjectUnsubscribe
>

Re: picoLisp 19.12: variable length array in structure fixes

[Mike]

hi all,

> If you are interested I have patched the 19.12 32bit sources to compile 
> without GCC.
> I have attached the changed files: pico.h, main.c, apply.c and flow.c

My testing status for https://github.com/picolisp/picolisp

1. pil @lib/test.l +
clang  - ok
clang+asan - ok
tcc-git- ok


2. huge.l - failed (you must take care)
$ git clone https://git.envs.net/mpech/tankf33der.git
$ cd tankf33der
$ pil huge.l + 
OK-UTF8
OK-Montgomery
OK-Leibniz
OK-math
OK-forth
OK-parse2list
OK-mapreduce
OK-pow
OK-destr
OK-test-car
OK-bitwise
# reduce redefined
# pow redefined
OK-4clojure
# pow redefined
// hangs in advent2015/code2015.l
// is it hangs or very-very slow?
// the same in crypto/test.l

!!!
Super goal - huge.l should pass all code with your patch set.


p.s. I dont have enough energy right now to debug all this.  

(mike)

--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: picoLisp 19.12: variable length array in structure fixes

[Alfonso Villén]

Hello,

with your patches, I've compiled Picolisp on 32-bit Termux + Clang without
any problems.

Thanks a lot! Great job!

Greetings
Alfonso V.

Andras Pahi  schrieb am Sa., 18. Apr. 2020, 20:00:

> Hello,
>
> If you are interested I have patched the 19.12 32bit sources to compile
> without GCC.
> I have attached the changed files: pico.h, main.c, apply.c and flow.c
>
> Since clang does not support variable length array in structures I
> allocate the bindFrame
> with alloca() and provided a macro in pico.h to ease this: allocFrame().
>
> I know that the 32bit version is not the mainstream version, but feel free
> to
> abuse the patches.
>
> Regards,
> Andras Pahi
>
>

Re: picoLisp 19.12: variable length array in structure fixes

[Alexander Burger]

Hi Andras,

> If you are interested I have patched the 19.12 32bit sources to compile 
> without GCC.
> I have attached the changed files: pico.h, main.c, apply.c and flow.c

Thanks a lot!


> Since clang does not support variable length array in structures I allocate 
> the bindFrame
> with alloca() and provided a macro in pico.h to ease this: allocFrame().
> 
> I know that the 32bit version is not the mainstream version, but feel free to
> abuse the patches.

Cool!

As I'm concentrating on pil21, I'm glad if development and maintenance of pil32,
mini and/or ersatz is taken care of by others. Until it is replaced by pil21
next year, I will do necessary fixes to pil64 and then - if all goes well -
completely replace it with pil21.

Let's hope that no major problems pop up ... ;)

☺/ A!ex


-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe