Re: The status of JIT compiler of Guile (was: Guile benchmark)

[Nala Ginrut]

This version has fixed file missing during compilation.


On Tue, Mar 14, 2017 at 10:35 AM, Nala Ginrut  wrote:
> Rebased to the latest, folks may download the tarball directly
> https://github.com/NalaGinrut/guile-tjit/releases/tag/tjit-2.1.8.975-1f6fc-rebase
>
>
> On Mon, Mar 6, 2017 at 7:19 PM, Nala Ginrut  wrote:
>> I've rebased the latest master (2.1.7.1040-0f7db-dirty) and fixed to work.
>> The result become more interesting with the naive case mentioned in my
>> earlier mail:
>> Chez: 15s
>> Guile-JIT: 12s
>>
>> Alright, we still need to face some problems.
>>
>> In Atsuro's original design, there're two files should be included
>> with Scheme's "include" function  in the compiling time. IMO this is
>> not a good way, since it'll assume the building path is the same with
>> source path. Then it complains jit-code-t.scm and
>> compat-x86_64-linux.scm are missing. I've fixed this issue in Makefile
>> to make a soft link. In the long term I think it's better to change
>> its design to use modules.
>>
>> And we don't have to generate jit-code-t.scm every time we compile.
>> This file is generated by gen-code-t.scm according to GNU-lightning
>> header file. Next step, I will move it to script directory and we may
>> generate it with guild. It's a standalone script actually.
>>
>> To be continued.

Re: The status of JIT compiler of Guile (was: Guile benchmark)

[Nala Ginrut]

Rebased to the latest, folks may download the tarball directly
https://github.com/NalaGinrut/guile-tjit/releases/tag/tjit-2.1.8.975-1f6fc-rebase


On Mon, Mar 6, 2017 at 7:19 PM, Nala Ginrut  wrote:
> I've rebased the latest master (2.1.7.1040-0f7db-dirty) and fixed to work.
> The result become more interesting with the naive case mentioned in my
> earlier mail:
> Chez: 15s
> Guile-JIT: 12s
>
> Alright, we still need to face some problems.
>
> In Atsuro's original design, there're two files should be included
> with Scheme's "include" function  in the compiling time. IMO this is
> not a good way, since it'll assume the building path is the same with
> source path. Then it complains jit-code-t.scm and
> compat-x86_64-linux.scm are missing. I've fixed this issue in Makefile
> to make a soft link. In the long term I think it's better to change
> its design to use modules.
>
> And we don't have to generate jit-code-t.scm every time we compile.
> This file is generated by gen-code-t.scm according to GNU-lightning
> header file. Next step, I will move it to script directory and we may
> generate it with guild. It's a standalone script actually.
>
> To be continued.

Re: The status of JIT compiler of Guile (was: Guile benchmark)

[Nala Ginrut]

I've rebased the latest master (2.1.7.1040-0f7db-dirty) and fixed to work.
The result become more interesting with the naive case mentioned in my
earlier mail:
Chez: 15s
Guile-JIT: 12s

Alright, we still need to face some problems.

In Atsuro's original design, there're two files should be included
with Scheme's "include" function  in the compiling time. IMO this is
not a good way, since it'll assume the building path is the same with
source path. Then it complains jit-code-t.scm and
compat-x86_64-linux.scm are missing. I've fixed this issue in Makefile
to make a soft link. In the long term I think it's better to change
its design to use modules.

And we don't have to generate jit-code-t.scm every time we compile.
This file is generated by gen-code-t.scm according to GNU-lightning
header file. Next step, I will move it to script directory and we may
generate it with guild. It's a standalone script actually.

To be continued.

The status of JIT compiler of Guile (was: Guile benchmark)

[Nala Ginrut]

Hi folks!
It's been a long time for me to be inactive here. ;-)

I saw there's thread to discuss Guile benchmark, I think it's better
to mention the JIT compiler of Guile. It exists and real, and of
course, very fast.

Thanks Atsuro Hoshino, who is the author of Guile JIT compiler.  We've
met on ICFP2016 in Nara. And I'm great impressed by this work. I'm
enjoying the contribution very much, especially when yesterday I made
it work successfully!

I think Atsuro is very busy these months, so that he has no time to
take a look on my patches. After waited half a year, I decide to take
the responsibility to maintain this project. I have to mention that
JIT compiler is not in Guile upstream repo at present, it's maintained
on GitHub.

I've forked the repo and patched it to make it work, and rebased it to
our latest master branch (2.1.7.980-f5904-dirty). Now it works on the
latest Guile code base. I'lI keep on maintaining it before Atsuro come
back to take it.
My plan is to keep it updating with the latest Guile master branch,
and try to fix problems, respond the issues, finally, when it becomes
mature, I'll send patches-set to Guile upstream.
Please come back soon, we need you, Atsuro ;-)

Here's my repo, reports and patches are welcome, I prefer maintain it
on GitHub, this will be easier for most of folks:
==
https://github.com/NalaGinrut/guile-tjit
==
Please follow nala-tjit branch which is maintaining by me.

To build it , you need gnu-lightning, and you should add
--enable-lightning when configure:
==
./configure --enable-lightning
==

To run it with JIT, please add --tjit option:
=
guile --tjit
=

The JIT is still experimental, so please report if you encounter any
problem, I'll try to help to fix it, and I'm here to invite
contributors to help this significant project. Atsuro's paper is here:
===
http://scheme2016.snow-fort.org/static/scheme16-paper3.pdf
===

About the performance, I tried a poor test:
code
 (define (fib n)
   (define (iter a b c)
 (cond ((= c 0) b)
   (#t (iter (+ a b) a (- c 1)
   (iter 1 0 n))

 (define a (fib 100))
=end

It's not a fair test, and I'm a bad benchmarker, but it shows
interesting result on my machine:
Chez: 15s
guile-master: 28s
guile-jit: 15s

Now you know it's different, huh?
Come to get involved please, try it on your code, and report
something, find bugs as possible. And let's pray it could be fixed
automatically. ;-P

Best regards.

Re: Guile benchmark

[Arne Babenhauserheide]

Chris Vine  writes:
> No, on my Haswell laptop running 64-bit linux, solving the 40,000th
> prime (479909) is 1.26 times faster in chez scheme than in guile-2.1.7,
> on the simple algorithm. (0.509 seconds versus 0.624 seconds, on a
> i7-4712HQ CPU @ 2.30GHz.)  This is only the time taken in the body of
> the calculation.  Start-up times and printing times and so forth are
> ignored.

Ah, ok. Thank you!

> I have just tried out the same code on an older Sandybridge desktop
> @3.30 GHz running 32-bit linux which I can access remotely, and that had
> chez scheme 1.7 times faster than guile-2.1.7 so it looks to be
> somewhat CPU and/or cache and/or pointer-width sensitive. Anyway, it is
> just one benchmark.
>
> I think I am about schemed out so installing Stalin is probably not an
> option.  Like you, I am told it is fast.  Guile's selling point IMO is
> its libraries and its FFI.  Guile 2.2 certainly seems adequately fast
> at run time.  Possibly compile times may be an issue, I don't know.

Factor 10 between C++ and pure Scheme is pretty awesome. Implementing
algorithms in pure Python carries a penalty of easily factor 100 (though
delegating anything expensive to builtin operators is fast).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature

Re: Guile benchmark

[Chris Vine]

On Tue, 28 Feb 2017 11:28:32 +0100
Arne Babenhauserheide  wrote:
> Chris Vine  writes:
> 
> > On Mon, 27 Feb 2017 21:00:54 +0100
> > Andy Wingo  wrote:  
> >> Hi,
> >> 
> >> On Thu 26 Jan 2017 09:39, Rchar  writes:  
> >> > https://ecraven.github.io/r7rs-benchmarks/benchmark.html
> >> > Is Guile slow or fast, comparing to others?
> >> 
> >> Schemes that compile to native code go faster.  Guile compiles to
> >> bytecode right now, so it's generally (though not always!) slower
> >> than the native-compiling schemes.  But compared to the
> >> bytecode-interpreter schemes it's pretty fast.  
> >
> > On reading this, out of interest I wrote a very simple program
> > solving primes, using the basic 'seive odd divisors to square root'
> > algorithm. I tested guile-2.0, guile-2.2, chicken and chez scheme
> > on it.
> >
> > guile-2.0 was a little slower than chicken with chicken compiled to
> > C, but guile-2.2 on that test took about 75% of the time of
> > chicken, and about 50% of the time of guile-2.0.  chez scheme was
> > fastest of all, taking about 50% of the time of chicken.  OK,
> > chicken may not be the fastest of "compile to C" schemes.  
> 
> Do I read this correctly that Chez took only about 30% less time than
> Guile 2.2?
> 
> Could you try stalin, too? (Chez wins 28 comparisons, Stalin 14, so
> that would be an obvious target)

I feared that my enumeration might confuse.

No, on my Haswell laptop running 64-bit linux, solving the 40,000th
prime (479909) is 1.26 times faster in chez scheme than in guile-2.1.7,
on the simple algorithm. (0.509 seconds versus 0.624 seconds, on a
i7-4712HQ CPU @ 2.30GHz.)  This is only the time taken in the body of
the calculation.  Start-up times and printing times and so forth are
ignored.

I have just tried out the same code on an older Sandybridge desktop
@3.30 GHz running 32-bit linux which I can access remotely, and that had
chez scheme 1.7 times faster than guile-2.1.7 so it looks to be
somewhat CPU and/or cache and/or pointer-width sensitive. Anyway, it is
just one benchmark.

I think I am about schemed out so installing Stalin is probably not an
option.  Like you, I am told it is fast.  Guile's selling point IMO is
its libraries and its FFI.  Guile 2.2 certainly seems adequately fast
at run time.  Possibly compile times may be an issue, I don't know.

Chris

Re: Guile benchmark

[Arne Babenhauserheide]

Chris Vine  writes:

> On Mon, 27 Feb 2017 21:00:54 +0100
> Andy Wingo  wrote:
>> Hi,
>> 
>> On Thu 26 Jan 2017 09:39, Rchar  writes:
>> > https://ecraven.github.io/r7rs-benchmarks/benchmark.html
>> > Is Guile slow or fast, comparing to others?  
>> 
>> Schemes that compile to native code go faster.  Guile compiles to
>> bytecode right now, so it's generally (though not always!) slower than
>> the native-compiling schemes.  But compared to the
>> bytecode-interpreter schemes it's pretty fast.
>
> On reading this, out of interest I wrote a very simple program solving
> primes, using the basic 'seive odd divisors to square root' algorithm.
> I tested guile-2.0, guile-2.2, chicken and chez scheme on it.
>
> guile-2.0 was a little slower than chicken with chicken compiled to C,
> but guile-2.2 on that test took about 75% of the time of chicken, and
> about 50% of the time of guile-2.0.  chez scheme was fastest of all,
> taking about 50% of the time of chicken.  OK, chicken may not be the
> fastest of "compile to C" schemes.

Do I read this correctly that Chez took only about 30% less time than
Guile 2.2?

Could you try stalin, too? (Chez wins 28 comparisons, Stalin 14, so that
would be an obvious target)

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature

Re: Guile benchmark

[Chris Vine]

On Mon, 27 Feb 2017 21:00:54 +0100
Andy Wingo  wrote:
> Hi,
> 
> On Thu 26 Jan 2017 09:39, Rchar  writes:
> 
> > I wanted to compare Guile scheme to other scheme implementations
> > and I found
> > this:https://ecraven.github.io/r7rs-benchmarks/benchmark.html
> >
> > Is Guile slow or fast, comparing to others?  
> 
> Besides what Mike said, there are two kinds of Schemes in that
> benchmark: the ones that compile to native machine code (ahead of
> time, either with their own compilers or by emitting C, or
> just-in-time), and the ones that run some kind of bytecode
> interpreter.
> 
> Schemes that compile to native code go faster.  Guile compiles to
> bytecode right now, so it's generally (though not always!) slower than
> the native-compiling schemes.  But compared to the
> bytecode-interpreter schemes it's pretty fast.
> 
> One day we'll have a native compiler and we can start playing
> benchmark games with the big kids :)

On reading this, out of interest I wrote a very simple program solving
primes, using the basic 'seive odd divisors to square root' algorithm.
I tested guile-2.0, guile-2.2, chicken and chez scheme on it.

guile-2.0 was a little slower than chicken with chicken compiled to C,
but guile-2.2 on that test took about 75% of the time of chicken, and
about 50% of the time of guile-2.0.  chez scheme was fastest of all,
taking about 50% of the time of chicken.  OK, chicken may not be the
fastest of "compile to C" schemes.

Having said that, all of them were considerably slower than the same
code written in native C++, probably because the compiler optimizes
better (it was about 5 times faster than chez scheme).  The performance
of clang and gcc was pretty much the same.

Solving primes is probably not particularly representative and I didn't
spend any more time with other benchmarks, so take it as you will. The
performance of guile-2.2 at run time was encouraging however.

Chris

Re: Guile benchmark

[Andy Wingo]

Hi,

On Thu 26 Jan 2017 09:39, Rchar  writes:

> I wanted to compare Guile scheme to other scheme implementations and I found 
> this:https://ecraven.github.io/r7rs-benchmarks/benchmark.html
>
> Is Guile slow or fast, comparing to others?

Besides what Mike said, there are two kinds of Schemes in that
benchmark: the ones that compile to native machine code (ahead of time,
either with their own compilers or by emitting C, or just-in-time), and
the ones that run some kind of bytecode interpreter.

Schemes that compile to native code go faster.  Guile compiles to
bytecode right now, so it's generally (though not always!) slower than
the native-compiling schemes.  But compared to the bytecode-interpreter
schemes it's pretty fast.

One day we'll have a native compiler and we can start playing benchmark
games with the big kids :)

Andy

Re: Guile benchmark

[Ludovic Courtès]

Hi,

Rchar  skribis:

> Is Guile scheme slows down entire GuixSD (If Guile speeds up, GuixSD also 
> speeds up)?

GuixSD is a GNU/Linux distro and even if some components are written in
Guile (such as the Shepherd and some low-level helpers), most of it is
written in C.

Guile’s speed has an influence on the speed of ‘guix’ commands though.

Ludo’.

Re: Guile benchmark

[Linas Vepstas]

On Thu, Jan 26, 2017 at 3:56 PM, Mike Gran  wrote:

>
> On Thursday, January 26, 2017 7:31 AM, Rchar  wrote:
>
>
>> Is Guile slow or fast, comparing to others?
>
> Guile is about average compared to the others. But it depends
> on the specific task. It is usually not the fastest or slowest.

FWIW, I have C++ code that has bindings both to guile and to python,
via cython.  The guile bindings were faster than python, which then
caused the cython maintainer to enter a nuclear arms race, to improve
performance, doing things like caching values in various places to improve
performance.  But then, one can do the same in guile, and it becomes a
game of who can be more clever in optimizing.

I do have one complaint: guile has a fairly hefty overhead in entering
and exiting: I forget now, and the measurements were done on a
circa-2011 PC, but I was getting something like 7K guile enter-exits
per second, which limits the speed at which you can cross the C/guile
boundary.   You can avoid this cost by entering guile and then caching,
but this requires some C-coding cleverness that naive users won't do.
(I'm currently getting maybe 500K/sec or 1M/sec guile-C crossings, but
only if I do this caching trick).

--linas