Re: Status of 64 picoLisp
Hi Alex, However, I am aware of the fact that in picoLisp the printing of numeric results is often much more expensive than the actual calculations, due to the conversions necessary for the decimal base. If I take out the final (prinl Y) from gmp-test2.l, I get: It makes a considerable difference. Interesting, it does not seem to make much difference with the C version when I take the final 'printf' out of the C code. So I believe that for practical uses, where processing does not consist solely of arithmetics, the overhead will be negligible, and not justify extra efforts. Yes, I haven't got any issues with bignum performance in picolisp;-) We are not using picolisp to compute Mersenne prime numbers http://www.mersenne.org/ after all:-D Cheers, Tomas -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Status of 64 picoLisp
Hi Tomas, Interesting, it does not seem to make much difference with the C version when I take the final 'printf' out of the C code. That's why I suspect that there might be a way to speed it up. On the other hand, bignum packages in C usually work with fixed length arrays (as opposed to linked lists of cells in picoLisp), which make things easier, and a lot more efficient. The picoLisp solution stays with the philosophy no limits, despite the fact that bignums above a certain size (a few thousand digits?) make no sense. Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Status of 64 picoLisp
Hi Tomas, it may be a bit late by now, but I'd like to write some comments about your posting last year, concerning bignum performance. On Thu, Oct 16, 2008 at 11:57:07AM +0100, Tomas Hlavaty wrote: I was curious to try picolisp bignums and must say that for somebody doing anything serious, it is probably rather inefficient. ... (= N 10) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log real 0m10.190s user 0m10.157s sys 0m0.008s Now I tried it here: $ time bin/picolisp gmp-test2.l /dev/null real0m11.938s user0m11.780s sys 0m0.160s As you see, with roughly the same result. However, I am aware of the fact that in picoLisp the printing of numeric results is often much more expensive than the actual calculations, due to the conversions necessary for the decimal base. If I take out the final (prinl Y) from gmp-test2.l, I get: $ time bin/picolisp gmp-test2.l real0m3.326s user0m3.200s sys 0m0.130s It makes a considerable difference. (= N 100) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log C-c C-cKilled real 17m58.856s user 17m51.687s sys 0m5.572s (killed after 18 mins!) Now if I also use 100 iterations (but without the printing to save time), I get: $ time bin/picolisp gmp-test2.l real5m4.904s user5m4.580s sys 0m0.320s This is still six times as long as the 50 seconds of your C program, but much better than the (unfinished) 18 minutes above. So I believe that for practical uses, where processing does not consist solely of arithmetics, the overhead will be negligible, and not justify extra efforts. If we'd want to optimize it, we should improve the decimal output conversion. Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Status of 64 picoLisp
On Tue, Oct 21, 2008 at 09:20:30AM +1100, konrad Zielinski wrote:
THe interesting thing will be handling C structs. As almost every
interesting C API makes heavy use of them.
Well, you know, I don't want to discuss such things prematurely. But as
you insist ... ;-)
Consulting my last year's notes, I think I had something like the
following in mind:
There is a function
(native fun lib ret val ..)
Actually, I don't like the name 'native'. Would prefer something
shorter. Perhaps just 'c' or 'C'?
fun is the name of a symbol in the library lib. It is located with
dlopen() and dlsym() as before.
ret and the vals specify the return value, followed by the
arguments. These structures are interpreted by 'native', building up
appropriate data structures.
C structures are defined as nested lists. I thought of the following
primitive types:
A address
B byte
C char
I int
L long
D double
NIL void
An uninitialized character array of 8 characters would be
(C . 8)
while an array of 8 zero characters is
(C 0 0 0 0 0 0 0 0)
That is, an atomic CDR is a size spec, while a list contains
one or more initializers.
A structure
struct {
int i[2];
double d;
char c[32];
};
would be written as
((I . 2) D (C . 32))
A function call to allocate four of these structures:
(setq
Stat '((I . 2) D (C . 32))
Ptr (native malloc libc.so A (I (* 4 (sizeof Stat )
So 'native' knows that it has to return a pointer 'A', and to pass a
single integer with the value (* 4 (sizeof Stat)) to 'malloc'.
'sizeof' is system dependent, knowing of the sizes of individual
primitive types.
Then we have a function 'struct' which takes a structure specification
and a memory pointer, and either stores an s-expression in memory (if a
third argument is given), or builds an s-expression from memory (if
not):
(struct Stat Ptr '((3 4) 123000 test))
(struct Stat Ptr) - ((3 4) 123000 test)
That all I have at the momen.
Cheers,
- Alex
--
UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
No matter how efficient or clever a virtual machine, it still requires additional steps in order to perform useful work. So there are really three efficient approaches to consider: 1. Accept that we have a ubiquitous x86(-64) mono-culture and primarily target that. I use PPC and ARM quite often, we almost went with Coldfire once, I know people working with MIPS and OpenRisc. 3. Recognise that established compilers do the work of optimisation, etc very well. In which case use the universal assembler as provided by the C compiler as our target. Consider the speed at which compilers like tinyCC do their work and the high levels of optimisation provided by GNU C and Intel C compilers. I probably should have mentioned that the LLVM project is not only a virtual machine, but an optimizing compiler from assembler to native machine code. Tinycc compiles fast, but does not produce very fast machine code. (It also is mostly limited to x86.) GCC is huge. It's interesting to note that Squeak (the latest incarnation of Smalltalk) was ported using a subset of Smalltalk called Slang in a similar manner to Alex's approach of using a generic assembler written in picoLisp. And PyPy is Python ported to a subset of Python. Recently they have seen the light and not only has a C backend, but an LLVM backend. OOPSLA'96 - Back to the Future This might be a superflous comment, but I remember seeing somewhere an assembler notation which was in effect a form of s-expr. It might not be s-expr, but LLVM assembler is very clean and is made to ease analysis and optimisation. I recommend: http://llvm.org/docs/LangRef.html it is a very fun read. About the comment made by Konrad Zielinski earlier in thread I suspect this would be in contradiction to some of the stated goals of PicoLisp, I can only say that there need not be a contradiction. The generic assembler can output LLVM IR as well as x86_64 code and ARM. It is just another target. regards, Jakob -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
On Wed, Oct 15, 2008 at 10:27:59PM +0100, Tomas Hlavaty wrote: your 64 bit Linux fine. You'll need to add the -m32 option to gcc.l .. or download the latest testing version. -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi Tomas, I was curious to try picolisp bignums and must say that for somebody doing anything serious, it is probably rather inefficient. As a I'm aware of that. The bignum implementation was not intended to be particularly fast, or - to put it corrrectly - cannot be expected to be very fast because numbers are implemented as linked lists instead of arrays. I believe the advantage of a singular internal structure (the cell) outweighs the speed disadvantage. Short numbers are sufficiently fast, but when numbers grow to a length of hundreds of cells, the speed will probably go down with the square of the length. For practical programs, I never experienced any bottlenecks due to arithmetic processing. Raw arithmetic speed is not of such importance in Lisp as in other languages, where most primitive operations (like array index calculations) depend on it. Would not it be better to use gmp library for bignums if they are going to be supported? For an application that needs the speed, why not? It is just a lot more trouble to support and use it. What is the reason picolisp has bignums in the first place? Do Being unlimited. One of the main goals of PicoLisp was that the programmer should never have to think about size limits (see doc/ref.html#intro). you/somebody else use it for anything? Would not it be simpler and I did need it for the RSA library lib/rsa.l when I used the Java Applet API. Anywah, on a 32bit system, 10 digits for short numbers are not enough for useful work. good enough on 64 bit systems not having them at all? Also on 64 bits I do not want to remember making sure not to cross the 18-digit border. With scaled fixed point arithmetics this can easily happen. In addition, the bignum structures are needed internally anyway, as the names of symbols are technically also bignums. In the 64 bit system, the CDR part of a number cell is used now, too, being more space-efficient. That is, a number is either a short number (60 bits + sign), or a cell with 64 bits in the CAR, and a number in the CDR. What impact on interfacing foreign libraries the asm rewrite have? It will still be possible to call external libraries. And also necessary, as the Posix system interface is used just as before, for I/O, memory and process management, networking etc. It is only the part that was written in C until now that is replaced. Cheers, - Alex -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
On Thu, Oct 16, 2008 at 10:24:40PM +1100, konrad Zielinski wrote: And now back to questions about 64 bit picolisp: Is switching to an assembler going to mean the demise of gcc.l ? Are we going to see inline picoLisp Assembler instead?O Yes, the current version of gcc.l will not work any longer :-( It is possible to design a similar mechanism, writing inline assembly instead of C, and calling 'as' instead of 'gcc'. But I'm not sure if this is the right way to go. In any case, I have a concept of a generic call to C functions in arbitrary external libraries. And I'm sure we will have plenty of other ideas when time comes ;-) How is a new assembler based version of Picolisp going to affect 32 bit platforms. I imagine they are going to be arround for quite a number of decades yet? Yes, I don't see 3.0 in production use for the near future, and I'll support both versions as long as they are needed. I can probably not switch all my customers to another version anyway. Cheers, - Alex -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi Alex, Yes, the current version of gcc.l will not work any longer :-( What is the reason for this not being possible? I though C and asm can be linked together (C is compiled to asm anyway). Cheers, Tomas -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi Tomas, Yes, the current version of gcc.l will not work any longer :-( What is the reason for this not being possible? I though C and asm can be linked together (C is compiled to asm anyway). On the instruction level this is correct, but the calling conventions of the (assembly) functions in the PicoLisp kernel are different. It is no longer possible to write a built-in function in C, as built-in functions expect their argument - and return the result - in the E register (%rbx on x86-64). And those functions need to be able to access the rest of the kernel for type checking, evaluation, garbage collection etc. In addition, such a function is required to start at an address which is a multiple of 16 plus 2 which is also difficult to achieve in C. As I wrote to Konrad, it might be possible design a similar mechanism, writing inline assembly instead of C, and calling 'as' instead of 'gcc', or invent other useful things. We should not worry about that too much now :-) Cheers, - Alex -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi All, It would appear that setting up a chroot is remarkably easy, well under debian anyway, I can't speak for other distros as I haven't tried. AndJus it seems to work quite nicely too, even if their are other ways to do it. Anyway now that I have a chroot I can install a version of Firefox with flash as well : ) Disclaimer: I have not examined the low level virtual machine to any level of detail. the following is a general observation about VM's. For all I know the people behind LLVM have done things better. Some languages are simply sufficiently different that they do not work well on some virtual machines. Attempting to implement Erlang on the JVM comes to mind. The underlying primities are different. Often what generic virtual machine means is good for any language sufficently close to C, IE staticly typed etc. Lisp is not decented from C and makes somwhat different assumptions about a lot of things, so it may not be a good fit. And now back to questions about 64 bit picolisp: Is switching to an assembler going to mean the demise of gcc.l ? Are we going to see inline picoLisp Assembler instead?O How is a new assembler based version of Picolisp going to affect 32 bit platforms. I imagine they are going to be arround for quite a number of decades yet? Regs Konrad On 16/10/2008, Tomas Hlavaty [EMAIL PROTECTED] wrote: Hi Alex, thanks for explanation. I was curious to try picolisp bignums and must say that for somebody doing anything serious, it is probably rather inefficient. As a benchmark, I tried the example from http://paste.lisp.org/display/15116 (setq X 0) (setq Y 1) (for (N 2 (= N 100) (inc N)) (let Z (+ X Y) (setq X Y) (setq Y Z))) (prinl Y) Very rough results using picolisp native bignums: (= N 1) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log real 0m0.131s user 0m0.124s sys 0m0.008s (= N 10) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log real 0m10.190s user 0m10.157s sys 0m0.008s (= N 100) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log C-c C-cKilled real 17m58.856s user 17m51.687s sys 0m5.572s (killed after 18 mins!) The original C program: $ time ./gmp gmp.log real 0m50.060s user 0m50.059s sys 0m0.004s I wrote simple ffi wrapper for gmp library and the results: $ time ../../p gmp-test.l -bye gmp-test.log real 0m50.507s user 0m50.239s sys 0m0.248s using the following code: (setq X (mpz_new)) (setq Y (mpz_new)) (mpz_init X) (mpz_init Y) (mpz_set_ui X 0) (mpz_set_ui Y 1) (setq Z (mpz_new)) (for (N 2 (= N 100) (inc N)) (mpz_init Z) (mpz_add Z X Y) (mpz_set X Y) (mpz_set Y Z) (mpz_clear Z)) (mpz_print Y) (prinl) Would not it be better to use gmp library for bignums if they are going to be supported? What is the reason picolisp has bignums in the first place? Do you/somebody else use it for anything? Would not it be simpler and good enough on 64 bit systems not having them at all? What impact on interfacing foreign libraries the asm rewrite have? Cheers, Tomas -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED] -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi Alex, thanks for explanation. I was curious to try picolisp bignums and must say that for somebody doing anything serious, it is probably rather inefficient. As a benchmark, I tried the example from http://paste.lisp.org/display/15116 (setq X 0) (setq Y 1) (for (N 2 (= N 100) (inc N)) (let Z (+ X Y) (setq X Y) (setq Y Z))) (prinl Y) Very rough results using picolisp native bignums: (= N 1) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log real0m0.131s user0m0.124s sys 0m0.008s (= N 10) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log real0m10.190s user0m10.157s sys 0m0.008s (= N 100) $ time ~/picolisp/p gmp-test2.l -bye gmp-test2.log C-c C-cKilled real17m58.856s user17m51.687s sys 0m5.572s (killed after 18 mins!) The original C program: $ time ./gmp gmp.log real0m50.060s user0m50.059s sys 0m0.004s I wrote simple ffi wrapper for gmp library and the results: $ time ../../p gmp-test.l -bye gmp-test.log real0m50.507s user0m50.239s sys 0m0.248s using the following code: (setq X (mpz_new)) (setq Y (mpz_new)) (mpz_init X) (mpz_init Y) (mpz_set_ui X 0) (mpz_set_ui Y 1) (setq Z (mpz_new)) (for (N 2 (= N 100) (inc N)) (mpz_init Z) (mpz_add Z X Y) (mpz_set X Y) (mpz_set Y Z) (mpz_clear Z)) (mpz_print Y) (prinl) Would not it be better to use gmp library for bignums if they are going to be supported? What is the reason picolisp has bignums in the first place? Do you/somebody else use it for anything? Would not it be simpler and good enough on 64 bit systems not having them at all? What impact on interfacing foreign libraries the asm rewrite have? Cheers, Tomas -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi Konrad, I was just wondering what the current status of 64 bit picolisp is. The good news: It is under work since more than one year now :-) The bad new: I'm afraid it will still take quite a while :-( I do not want to go public with it before I'm sure that most details are settled down. But I already have a skeleton running, a proof of concept. The basic subsystems are ready so far (eval, apply, symbol tables, exception handling, garbage collection), but the huge amount of I/O functionalities is still missing. Currently I'm working on the bignum arithmetics. It is a complete rewrite. Even the implementation language changed. Instead of C it is written in a generic assembler (which in turn is written in PicoLisp :) that generates GNU assembler code (currently there is only a x86-64 generator, but other CPUs are possible). The problem is that I can work on it only at my spare time, and this is usually scarce. Things would be different if I'd get paid for it ;-) I hope I can show a protype during the next year. I;m on a 64 bit system which for the moment has prevented me from trying out Thomas's Async read and write code. Is it a non-x86-64 system, preventing you from running 32bit binaries? Does the larger cell size mean that there is an extra unneeded bit on every pointer, which could be used as an additional flag bit. Exact. For say floating point numbers? No, but short and big numbers will be separate types internally. The structure model is cnt S010 big S100 sym 1000 cell Floating point numbers would have been an option, but are not intended. I ask the latter as this is the one of the two points I could see Picolisp being flamed for, when it gets sufficently popular. The other is use of dynamic scope, though i suspect we agree on this one, its a design decision live with it. Yep. As you know, I'm convinced that dynamic binding is far superior to lexical binding ;-) Cheers, - Alex -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
On 15 Oct 2008, at 9:11 AM, Alexander Burger wrote: It is a complete rewrite. Even the implementation language changed. Instead of C it is written in a generic assembler (which in turn is written in PicoLisp :) that generates GNU assembler code (currently there is only a x86-64 generator, but other CPUs are possible). Hmm. Beware of the second-system effect. :) John -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
On 15 Oct 2008, at 9:11 AM, Alexander Burger wrote: It is a complete rewrite. Even the implementation language changed. Instead of C it is written in a generic assembler (which in turn is written in PicoLisp :) that generates GNU assembler code (currently there is only a x86-64 generator, but other CPUs are possible). Oh. Please consider http://en.wikipedia.org/wiki/Low_Level_Virtual_Machine Because it is awesome, and if you port to that, you do not have to port to any other architecture. I was only recently made aware of LLVM, and it is really cool. Apple reportedly has started using it, probably wise from their past experience with switching CPU architecture. With LLVM, switch as much as you like. And no, LLVM is NOT another Java in disguise. LLVM is like a real CPU. regards, Jakob -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi Konrad, I;m on a 64 bit system which for the moment has prevented me from trying out Thomas's Async read and write code. I'm working on a 32 bit chroot enviornment to run pico in for now, but it would be nice to go native. It should work even on 64 bit Linux without chroot environment. You'll need to install some 32 bit compatibility packages and gcc, I think these were called ia32-libs, libc6-dev-i386 and maybe something more. Then -m32 will instruct the compiler and linker to generate use 32 bit code and use 32 bit libraries. The compiled application will run on your 64 bit Linux fine. You'll need to add the -m32 option to gcc.l though, or use the 'patch' trick Alex recommended in that mail thread about async io. Cheers, Tomas -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
Hi Alex, It is a complete rewrite. Even the implementation language changed. Instead of C it is written in a generic assembler (which in turn is written in PicoLisp :) that generates GNU assembler code (currently there is only a x86-64 generator, but other CPUs are possible). I guess that miniPicoLisp is not 64 bit incarnation of future picoLisp-3 then? What are the reasons for a) complete rewrite and b) switching from C to asm? Cheers, Tomas -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
Re: Status of 64 picoLisp
I suspect this would be in contradiction to some of the stated goals of PicoLisp, For one you would no longer be close to the machine. Just close to the virtual machine. On 16/10/2008, Jakob [EMAIL PROTECTED] wrote: On 15 Oct 2008, at 9:11 AM, Alexander Burger wrote: It is a complete rewrite. Even the implementation language changed. Instead of C it is written in a generic assembler (which in turn is written in PicoLisp :) that generates GNU assembler code (currently there is only a x86-64 generator, but other CPUs are possible). Oh. Please consider http://en.wikipedia.org/wiki/Low_Level_Virtual_Machine Because it is awesome, and if you port to that, you do not have to port to any other architecture. I was only recently made aware of LLVM, and it is really cool. Apple reportedly has started using it, probably wise from their past experience with switching CPU architecture. With LLVM, switch as much as you like. And no, LLVM is NOT another Java in disguise. LLVM is like a real CPU. regards, Jakob -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED] -- UNSUBSCRIBE: mailto:[EMAIL PROTECTED]
