Hello Sven,
I went and removed the Stdouts that you mention and other timing code
from the loops.
I am running the test now, to see if that makes much difference. I do
not think it will.
The reason I put that in there is because it take so long to run. It can
be frustrating to wait and wait and not know if your test is doing
anything or not. So I put the code in to let me know.
One of your parameters is incorrect. It is 100 iterations not 10.
I learned early on in this experiment that I have to do a large number
of iterations or C, C++, Java, etc are too fast to have comprehensible
results.
I can tell if any of the implementations is incorrect by the final nsum.
All implementations must produce the same result.
Thanks for the comments.
Jimmie
On 1/7/22 07:40, Sven Van Caekenberghe wrote:
Hi Jimmie,
I loaded your code in Pharo 9 on my MacBook Pro (Intel i5) macOS 12.1
I commented out the Stdio logging from the 2 inner loops (#loop1, #loop2) (not
done in Python either) as well as the MessageTally spyOn: from #run (slows
things down).
Then I ran your code with:
[ (LanguageTest newSize: 60*24*5*4 iterations: 10) run ] timeToRun.
which gave me "0:00:09:31.338"
The console output was:
===
Starting test for array size: 28800 iterations: 10
Creating array of size: 28800 timeToRun: 0:00:00:00.031
Starting loop 1 at: 2022-01-07T14:10:35.395394+01:00
Loop 1 time: nil
nsum: 11234.235001659388
navg: 0.39007760422428434
Starting loop 2 at: 2022-01-07T14:15:22.108433+01:00
Loop 2 time: 0:00:04:44.593
nsum: 11245.697629561537
navg: 0.3904756121375534
End of test. TotalTime: 0:00:09:31.338
===
Which would be twice as fast as Python, if I got the parameters correct.
Sven
On 7 Jan 2022, at 13:19, Jimmie Houchin <[email protected]> wrote:
As I stated this is a micro benchmark and very much not anything resembling a
real app, Your comments are true if you are writing your app. But if you want
to stress the language you are going to do things which are seemingly non-sense
and abusive.
Also as I stated. The test has to be sufficient to stress faster languages or
it is meaningless.
If I remove the #sum and the #average calls from the inner loops, this is what
we get.
Julia 0.2256 seconds
Python 5.318 seconds
Pharo 3.5 seconds
This test does not sufficiently stress the language. Nor does it provide any
valuable insight into summing and averaging which is done a lot, in lots of
places in every iteration.
If you notice that inner array changes the array every iteration. So every call
to #sum and #average is getting different data.
Full Test
Julia 1.13 minutes
Python 24.02 minutes
Pharo 2:09:04
Code for the above is now published. You can let me know if I am doing
something unequal to the various languages.
And just remember anything you do which sufficiently changes the test has to be
done in all the languages to give a fair test. This isn't a lets make Pharo
look good test. I do want Pharo to look good, but honestly.
Yes, I know that I can bind to BLAS or other external libraries. But that is
not a test of Pharo. The Python is plain Python3 no Numpy, just using the the
default list [] for the array.
Julia is a whole other world. It is faster than Numpy. This is their domain and
they optimize, optimize, optimize all the math. In fact they have reached the
point that some pure Julia code beats pure Fortran.
In all of this I just want Pharo to do the best it can.
With the above results unless you already had an investment in Pharo, you
wouldn't even look. :(
Thanks for exploring this with me.
Jimmie
On 1/6/22 18:24, John Brant wrote:
On Jan 6, 2022, at 4:35 PM, Jimmie Houchin <[email protected]> wrote:
No, it is an array of floats. The only integers in the test are in the indexes
of the loops.
Number random. "generates a float 0.8188008774329387"
So in the randarray below it is an array of 28800 floats.
It just felt so wrong to me that Python3 was so much faster. I don't care if
Nim, Crystal, Julia are faster. But...
I am new to Iceberg and have never shared anything on Github so this is all new
to me. I uploaded my language test so you can see what it does. It is a
micro-benchmark. It does things that are not realistic in an app. But it does
stress a language in areas important to my app.
https://github.com/jlhouchin/LanguageTestPharo
Let me know if there is anything else I can do to help solve this problem.
I am a lone developer in my spare time. So my apologies for any ugly code.
Are you sure that you have the same algorithm in Python? You are calling sum
and average inside the loop where you are modifying the array:
1 to: nsize do: [ :j || n |
n := narray at: j.
narray at: j put: (self loop1calc: i j: j n: n).
nsum := narray sum.
navg := narray average ]
As a result, you are calculating the sum of the 28,800 size array 28,800 times (plus
another 28,800 times for the average). If I write a similar loop in Python, it looks
like it would take almost 9 minutes on my machine without using numpy to calculate
the sum. The Pharo code takes ~40 seconds. If this is really how the code should be,
then I would change it to not call sum twice (once for sum and once in average). This
will almost result in a 2x speedup. You could also modify the algorithm to update the
nsum value in the loop instead of summing the array each time. I think the updating
would require <120,000 math ops vs the >1.6 billion that you are performing.
John Brant
