Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

Thanks, Stefan, it's good to know this already exists! Seeing it mentioned 
side-by-side with an Bloomberg API indicates that this feature request 
probably came from the financial industry.

-Zhong

On Tuesday, August 2, 2016 at 11:50:51 PM UTC-5, Stefan Karpinski wrote:
>
> Julia Computing offers a product (JuliaInXL) which does exactly this.
>
> On Tue, Aug 2, 2016 at 11:07 PM, Zhong Pan  > wrote:
>
>> Eric, hustf,
>>
>> I think making Julia attractive to Excel/VBA users will be quite 
>> valuable. Excel still rules in business world for simple to moderately 
>> complex data analysis. Strangely, even engineers love it - there is still a 
>> large group of hardware/mechanical engineers who are not productive in a 
>> general purpose programming language, but they love Excel for its 
>> simplicity, programmability, and visual appeal (who doesn't like cells with 
>> dizzying colors and fancy fonts :-)). And I know some engineers who can 
>> program really well actually wrote quite sophisticated VBA macros inside 
>> Excel so the resulted sheet can be used intuitively by a non-programmer 
>> (e.g. a manufacturing staff or a field support) while saving the trouble of 
>> developing an independent GUI which will take an extra head count. 
>>
>> While Julia will certainly win the speed contest, I don't think Julia 
>> should or could replace Excel/VBA. Maybe a more practical and gentle 
>> approach is to make Julia conveniently available from inside Excel/VBA, 
>> similar to what RExcel  does? When 
>> the complexity or computational load run out of VBA's capability, the 
>> Exceler can trust Julia to get the job done, fast.
>>
>> -Zhong
>>
>>
>> On Monday, August 1, 2016 at 1:14:37 AM UTC-5, Eric Forgy wrote:
>>>
>>> I mentioned to Prof. Edelman (only half jokingly) at an event in 
>>> Singapore, that we should add Excel/VBA to the list of benchmarks. 
>>>
>>> If I'm in a corporate setting and trying to sell Julia for some internal 
>>> project, the person making the call has probably never heard of any of the 
>>> languages in the Julia benchmark, but they have heard of Excel/VBA, so, as 
>>> silly as it may seem, I actually think it could go a long way for Julia 
>>> evangelists to see more comparisons to Excel/VBA.
>>>
>>> On Monday, August 1, 2016 at 1:45:24 AM UTC+8, hustf wrote:

 It is nice to have a little check on speed from time to time. I still 
 use VBA for easy cooperation with less programming savvy colleguaes.

 Julia 1.17s.
 VBA (excel alt + f11):12 s.

 This is a bit unfair to neolithic man Joel Spolsky since no 
 optimization was performed:

 Sub benchmark()
 nsamples = 100
 Dim y() As Double
 ReDim y(1 To nsamples)
 x = y
 For i = 1 To nsamples
 x(i) = (i - 1) * 5 / (nsamples - 1)
 Next
 Debug.Print ("\nBrutal-force loops, 100 times:")
 sngtime = Timer
 For m = 1 To 100
 For n = 1 To nsamples
 y(n) = Cos(2 * x(n) + 5)
 Next
 Next
 Debug.Print Timer - sngtime
 End Sub


>

Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[hustf]

Zhong, people chop wood and they play the piano, too. Fill in Julia or 
Excel where you like. 

Sisyphuss, VBA show a 4% speed GAIN through making nsamples and y() public 
variables. Try it out!

Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Stefan Karpinski]

Julia Computing offers a product (JuliaInXL) which does exactly this.

On Tue, Aug 2, 2016 at 11:07 PM, Zhong Pan  wrote:

> Eric, hustf,
>
> I think making Julia attractive to Excel/VBA users will be quite valuable.
> Excel still rules in business world for simple to moderately complex data
> analysis. Strangely, even engineers love it - there is still a large group
> of hardware/mechanical engineers who are not productive in a general
> purpose programming language, but they love Excel for its simplicity,
> programmability, and visual appeal (who doesn't like cells with dizzying
> colors and fancy fonts :-)). And I know some engineers who can program
> really well actually wrote quite sophisticated VBA macros inside Excel so
> the resulted sheet can be used intuitively by a non-programmer (e.g. a
> manufacturing staff or a field support) while saving the trouble of
> developing an independent GUI which will take an extra head count.
>
> While Julia will certainly win the speed contest, I don't think Julia
> should or could replace Excel/VBA. Maybe a more practical and gentle
> approach is to make Julia conveniently available from inside Excel/VBA,
> similar to what RExcel  does? When
> the complexity or computational load run out of VBA's capability, the
> Exceler can trust Julia to get the job done, fast.
>
> -Zhong
>
>
> On Monday, August 1, 2016 at 1:14:37 AM UTC-5, Eric Forgy wrote:
>>
>> I mentioned to Prof. Edelman (only half jokingly) at an event in
>> Singapore, that we should add Excel/VBA to the list of benchmarks.
>>
>> If I'm in a corporate setting and trying to sell Julia for some internal
>> project, the person making the call has probably never heard of any of the
>> languages in the Julia benchmark, but they have heard of Excel/VBA, so, as
>> silly as it may seem, I actually think it could go a long way for Julia
>> evangelists to see more comparisons to Excel/VBA.
>>
>> On Monday, August 1, 2016 at 1:45:24 AM UTC+8, hustf wrote:
>>>
>>> It is nice to have a little check on speed from time to time. I still
>>> use VBA for easy cooperation with less programming savvy colleguaes.
>>>
>>> Julia 1.17s.
>>> VBA (excel alt + f11):12 s.
>>>
>>> This is a bit unfair to neolithic man Joel Spolsky since no optimization
>>> was performed:
>>>
>>> Sub benchmark()
>>> nsamples = 100
>>> Dim y() As Double
>>> ReDim y(1 To nsamples)
>>> x = y
>>> For i = 1 To nsamples
>>> x(i) = (i - 1) * 5 / (nsamples - 1)
>>> Next
>>> Debug.Print ("\nBrutal-force loops, 100 times:")
>>> sngtime = Timer
>>> For m = 1 To 100
>>> For n = 1 To nsamples
>>> y(n) = Cos(2 * x(n) + 5)
>>> Next
>>> Next
>>> Debug.Print Timer - sngtime
>>> End Sub
>>>
>>>

Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

Eric, hustf,

I think making Julia attractive to Excel/VBA users will be quite valuable. 
Excel still rules in business world for simple to moderately complex data 
analysis. Strangely, even engineers love it - there is still a large group 
of hardware/mechanical engineers who are not productive in a general 
purpose programming language, but they love Excel for its simplicity, 
programmability, and visual appeal (who doesn't like cells with dizzying 
colors and fancy fonts :-)). And I know some engineers who can program 
really well actually wrote quite sophisticated VBA macros inside Excel so 
the resulted sheet can be used intuitively by a non-programmer (e.g. a 
manufacturing staff or a field support) while saving the trouble of 
developing an independent GUI which will take an extra head count. 

While Julia will certainly win the speed contest, I don't think Julia 
should or could replace Excel/VBA. Maybe a more practical and gentle 
approach is to make Julia conveniently available from inside Excel/VBA, 
similar to what RExcel  does? When 
the complexity or computational load run out of VBA's capability, the 
Exceler can trust Julia to get the job done, fast.

-Zhong

On Monday, August 1, 2016 at 1:14:37 AM UTC-5, Eric Forgy wrote:
>
> I mentioned to Prof. Edelman (only half jokingly) at an event in 
> Singapore, that we should add Excel/VBA to the list of benchmarks. 
>
> If I'm in a corporate setting and trying to sell Julia for some internal 
> project, the person making the call has probably never heard of any of the 
> languages in the Julia benchmark, but they have heard of Excel/VBA, so, as 
> silly as it may seem, I actually think it could go a long way for Julia 
> evangelists to see more comparisons to Excel/VBA.
>
> On Monday, August 1, 2016 at 1:45:24 AM UTC+8, hustf wrote:
>>
>> It is nice to have a little check on speed from time to time. I still use 
>> VBA for easy cooperation with less programming savvy colleguaes.
>>
>> Julia 1.17s.
>> VBA (excel alt + f11):12 s.
>>
>> This is a bit unfair to neolithic man Joel Spolsky since no optimization 
>> was performed:
>>
>> Sub benchmark()
>> nsamples = 100
>> Dim y() As Double
>> ReDim y(1 To nsamples)
>> x = y
>> For i = 1 To nsamples
>> x(i) = (i - 1) * 5 / (nsamples - 1)
>> Next
>> Debug.Print ("\nBrutal-force loops, 100 times:")
>> sngtime = Timer
>> For m = 1 To 100
>> For n = 1 To nsamples
>> y(n) = Cos(2 * x(n) + 5)
>> Next
>> Next
>> Debug.Print Timer - sngtime
>> End Sub
>>
>>

Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Sisyphuss]

Julia is not as "mature" as VBA, which prevents "analysts" of large firms 
adopting it.

In addition, they will be happier to continue using global variables.



On Monday, August 1, 2016 at 8:14:37 AM UTC+2, Eric Forgy wrote:
>
> I mentioned to Prof. Edelman (only half jokingly) at an event in 
> Singapore, that we should add Excel/VBA to the list of benchmarks. 
>
> If I'm in a corporate setting and trying to sell Julia for some internal 
> project, the person making the call has probably never heard of any of the 
> languages in the Julia benchmark, but they have heard of Excel/VBA, so, as 
> silly as it may seem, I actually think it could go a long way for Julia 
> evangelists to see more comparisons to Excel/VBA.
>
> On Monday, August 1, 2016 at 1:45:24 AM UTC+8, hustf wrote:
>>
>> It is nice to have a little check on speed from time to time. I still use 
>> VBA for easy cooperation with less programming savvy colleguaes.
>>
>> Julia 1.17s.
>> VBA (excel alt + f11):12 s.
>>
>> This is a bit unfair to neolithic man Joel Spolsky since no optimization 
>> was performed:
>>
>> Sub benchmark()
>> nsamples = 100
>> Dim y() As Double
>> ReDim y(1 To nsamples)
>> x = y
>> For i = 1 To nsamples
>> x(i) = (i - 1) * 5 / (nsamples - 1)
>> Next
>> Debug.Print ("\nBrutal-force loops, 100 times:")
>> sngtime = Timer
>> For m = 1 To 100
>> For n = 1 To nsamples
>> y(n) = Cos(2 * x(n) + 5)
>> Next
>> Next
>> Debug.Print Timer - sngtime
>> End Sub
>>
>>

Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[hustf]

It is nice to have a little check on speed from time to time. I still use 
VBA for easy cooperation with less programming savvy colleguaes.

Julia 1.17s.
VBA (excel alt + f11):12 s.

This is a bit unfair to neolithic man Joel Spolsky since no optimization 
was performed:

Sub benchmark()
nsamples = 100
Dim y() As Double
ReDim y(1 To nsamples)
x = y
For i = 1 To nsamples
x(i) = (i - 1) * 5 / (nsamples - 1)
Next
Debug.Print ("\nBrutal-force loops, 100 times:")
sngtime = Timer
For m = 1 To 100
For n = 1 To nsamples
y(n) = Cos(2 * x(n) + 5)
Next
Next
Debug.Print Timer - sngtime
End Sub

Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[dextorious]

Since I contributed the Numba JIT timing earlier in the thread, it seems 
only fair to note that the modified Julia version with the properly 
preallocated data is now 17% faster than the Numba version on my computer. 
Overall, this seems to support my thesis that good Julia code is on par or 
slightly faster than the Python code using the JIT libraries (Numba, Hope, 
Parakeet, etc.) with the added benefit of not requiring external libraries. 
I've also seen some cases where the difference is even more pronounced in 
the favor of Julia because of the ability to use @simd to prod the LLVM 
vectorizer along and the ability to exert precise control over memory 
allocations, but this is highly use case specific.

On Tuesday, July 26, 2016 at 12:47:53 AM UTC+3, Tim Holy wrote:
>
> Given the apparent interest in the topic and the decisions that people 
> seem to 
> be making, it seems worth pointing out that folks are still using 
> apples-to- 
> oranges comparisons on this benchmark. 
>
> There are at least two important differences: 
> - in the other languages, `linspace` allocates a vector, but in Julia it 
> (currently) creates a compact object from which values are computed 
> on-the-fly. 
> That computation involves a division, and division is slow. 
> - The languages like C aren't doing bounds-checking. You might imagine 
> adding 
> `@inbounds` to the Julia version. But `@inbounds` has no impact on 
> LinSpace 
> objects in julia 0.4. In julia 0.5, it does work like you'd expect 
> (thanks, 
> Blake Johnson). 
>
> Combining these observations, we can `collect` the values into a `Vector` 
> and 
> then use `@inbounds`. For me the version below is nearly twice as fast as 
> the 
> original: 
>
> function benchmark() 
> nsamples = 100 
> x = collect(linspace(0, 5, nsamples)) 
> y = zeros(nsamples) 
> # attempt to trigger JIT to compile all functions needed in the loops 
> # before profiling 
> a = cos(0.0); a = 1.5*2.5; a = 1.5+2.5; 
> println("\nBrutal-force loops, 100 times:") 
> @time begin 
> for m = 1:100 
> @inbounds for n = 1:nsamples 
> y[n] = cos(2*x[n]+5); 
> end 
> end 
> end 
> end 
> benchmark(); 
>
> Best, 
> --Tim 
>
> On Monday, July 25, 2016 2:02:41 PM CDT Zhong Pan wrote: 
> > Agree that while raw speed is important, in most situations it wouldn't 
> be 
> > the most important reason to choose one programming language over 
> another. 
> > 
> > I came from the angle of an engineer in a small company. For myself, the 
> > main attraction of Julia was the easiness to achieve decent speed 
> without 
> > making much explicit effort: that means what feels more natural 
> vectorized 
> > will be vectorized, while what feels more natural in a loop will be in a 
> > loop; that means I don't need to resort to another language or a library 
> > only for improving speed; and that means apart from sticking to a couple 
> > good habits, I can use objects, functions etc. the same way inside a 
> loop 
> > vs. outside. None of these is critical by itself, but they add up to an 
> > uninterrupted flow of thoughts while writing code to explore, try, fail, 
> > and retry, for many iterations. 
> > 
> > During this "less careful" prototyping, 1-2x slow down is fine, but with 
> > Julia I know I won't sit there for tens of minutes waiting for a result 
> > while debating myself whether I should rewrite it in C++ or rehaul the 
> code 
> > with Cython etc.; instead I can rest assured that as long as my algo and 
> > coding have no mistakes or major flaws, the speed is close to what I 
> will 
> > get even if I make several times more effort to rewrite it in C++. 
> > 
> > Another big deal for me is the resulted removal of the barrier between 
> > prototype and production code. For production I can review and improve 
> my 
> > code carefully, but rewriting it in a less expressive language is too 
> much. 
> > 
> > I was a huge fan of Python (heck I even persuaded my previous boss, a 
> VP, 
> > to pick up Python - though I don't know if he really had time to finish 
> it. 
> > 
> > :-)). However, the slow raw speed and the over-freedom to change class 
> > 
> > definition anywhere always gave me the itch to find something better. My 
> > brother at JPL who worked on Python projects also complained about 
> having 
> > to think really hard to vectorize almost everything and then couldn't 
> > easily understand what he was doing a few months later because the code 
> was 
> > too unnatural for the problem; the indentation was also a big headache 
> as 
> > collaborators use different editors with different tab definitions. 
> > 
> > So I'm really happy to have found Julia, which gave me the same joy as 
> > coding in Python and removed the main itches. 
> > 
> > -Zhong 
>
>
>

Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Tim Holy]

Given the apparent interest in the topic and the decisions that people seem to 
be making, it seems worth pointing out that folks are still using apples-to-
oranges comparisons on this benchmark.

There are at least two important differences:
- in the other languages, `linspace` allocates a vector, but in Julia it 
(currently) creates a compact object from which values are computed on-the-fly. 
That computation involves a division, and division is slow.
- The languages like C aren't doing bounds-checking. You might imagine adding 
`@inbounds` to the Julia version. But `@inbounds` has no impact on LinSpace 
objects in julia 0.4. In julia 0.5, it does work like you'd expect (thanks, 
Blake Johnson).

Combining these observations, we can `collect` the values into a `Vector` and 
then use `@inbounds`. For me the version below is nearly twice as fast as the 
original:

function benchmark()
nsamples = 100
x = collect(linspace(0, 5, nsamples))
y = zeros(nsamples)
# attempt to trigger JIT to compile all functions needed in the loops
# before profiling
a = cos(0.0); a = 1.5*2.5; a = 1.5+2.5;
println("\nBrutal-force loops, 100 times:")
@time begin
for m = 1:100
@inbounds for n = 1:nsamples
y[n] = cos(2*x[n]+5);
end
end
end
end
benchmark();

Best,
--Tim

On Monday, July 25, 2016 2:02:41 PM CDT Zhong Pan wrote:
> Agree that while raw speed is important, in most situations it wouldn't be
> the most important reason to choose one programming language over another.
> 
> I came from the angle of an engineer in a small company. For myself, the
> main attraction of Julia was the easiness to achieve decent speed without
> making much explicit effort: that means what feels more natural vectorized
> will be vectorized, while what feels more natural in a loop will be in a
> loop; that means I don't need to resort to another language or a library
> only for improving speed; and that means apart from sticking to a couple
> good habits, I can use objects, functions etc. the same way inside a loop
> vs. outside. None of these is critical by itself, but they add up to an
> uninterrupted flow of thoughts while writing code to explore, try, fail,
> and retry, for many iterations.
> 
> During this "less careful" prototyping, 1-2x slow down is fine, but with
> Julia I know I won't sit there for tens of minutes waiting for a result
> while debating myself whether I should rewrite it in C++ or rehaul the code
> with Cython etc.; instead I can rest assured that as long as my algo and
> coding have no mistakes or major flaws, the speed is close to what I will
> get even if I make several times more effort to rewrite it in C++.
> 
> Another big deal for me is the resulted removal of the barrier between
> prototype and production code. For production I can review and improve my
> code carefully, but rewriting it in a less expressive language is too much.
> 
> I was a huge fan of Python (heck I even persuaded my previous boss, a VP,
> to pick up Python - though I don't know if he really had time to finish it.
> 
> :-)). However, the slow raw speed and the over-freedom to change class
> 
> definition anywhere always gave me the itch to find something better. My
> brother at JPL who worked on Python projects also complained about having
> to think really hard to vectorize almost everything and then couldn't
> easily understand what he was doing a few months later because the code was
> too unnatural for the problem; the indentation was also a big headache as
> collaborators use different editors with different tab definitions.
> 
> So I'm really happy to have found Julia, which gave me the same joy as
> coding in Python and removed the main itches.
> 
> -Zhong

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

Agree that while raw speed is important, in most situations it wouldn't be 
the most important reason to choose one programming language over another.

I came from the angle of an engineer in a small company. For myself, the 
main attraction of Julia was the easiness to achieve decent speed without 
making much explicit effort: that means what feels more natural vectorized 
will be vectorized, while what feels more natural in a loop will be in a 
loop; that means I don't need to resort to another language or a library 
only for improving speed; and that means apart from sticking to a couple 
good habits, I can use objects, functions etc. the same way inside a loop 
vs. outside. None of these is critical by itself, but they add up to an 
uninterrupted flow of thoughts while writing code to explore, try, fail, 
and retry, for many iterations.

During this "less careful" prototyping, 1-2x slow down is fine, but with 
Julia I know I won't sit there for tens of minutes waiting for a result 
while debating myself whether I should rewrite it in C++ or rehaul the code 
with Cython etc.; instead I can rest assured that as long as my algo and 
coding have no mistakes or major flaws, the speed is close to what I will 
get even if I make several times more effort to rewrite it in C++.

Another big deal for me is the resulted removal of the barrier between 
prototype and production code. For production I can review and improve my 
code carefully, but rewriting it in a less expressive language is too much.

I was a huge fan of Python (heck I even persuaded my previous boss, a VP, 
to pick up Python - though I don't know if he really had time to finish it. 
:-)). However, the slow raw speed and the over-freedom to change class 
definition anywhere always gave me the itch to find something better. My 
brother at JPL who worked on Python projects also complained about having 
to think really hard to vectorize almost everything and then couldn't 
easily understand what he was doing a few months later because the code was 
too unnatural for the problem; the indentation was also a big headache as 
collaborators use different editors with different tab definitions. 

So I'm really happy to have found Julia, which gave me the same joy as 
coding in Python and removed the main itches.

-Zhong

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[dextorious]

I haven't done any systematic benchmarking since Numba introduced the 
ability to JIT compile entire classes. In my experience, very well written 
Julia code is usually equivalent or better (in cases when @simd is helpful) 
compared to Numba JIT'd code. The Python code is sometimes easier to write, 
since Numba takes care of everything, but it's a double edged sword - if 
you run into a case where Numba doesn't work well or at all, you're just 
out of luck. In my personal view, the availability of Numba and other 
libraries just means Python vs Julia performance comparisons aren't 
particularly relevant, you should pick the language you prefer, not because 
you think it's faster, whereas if absolute performance is the only metric, 
you have to resort to Fortran/C++ anyway. I'm writing most of my code in 
Julia because I prefer the type system and multiple dispatch to Python's 
OOP, but I don't think it's meaningfully faster as long as you're willing 
to use appropriate libraries.

The one outlier is the fact that GPU programming is much easier in Python 
at the moment. Hopefully that will change soon, some of the progress in 
ArrayFire.jl and CUDA libraries is very promising.

On Monday, July 25, 2016 at 7:34:04 PM UTC+3, dnm wrote:
>
> Interesting. Did you use the updated Julia code? 
>
> Have you done any comparisons between reading and writing  Numba JIT 
> classes and Julia types in tight loops?
>
> On Monday, July 25, 2016 at 10:41:48 AM UTC-4, dexto...@gmail.com wrote:
>>
>> Just for the sake of comprehensiveness, I ran your Python benchmark 
>> through the Numba JIT library (which uses the same underlying LLVM 
>> infrastructure that Julia does) and on my computer the Python code is 
>> faster than Julia by 68%. Vanilla CPython is terrible for this kind of 
>> simple explicit loop code, but Numba and other JIT libraries largely solve 
>> that issue with minimal effort as long as the code is simple enough. That 
>> by no means solves all of Python's issues in the context of numerical 
>> programming and I'm sure the Julia benchmark could be improved as others 
>> have already mentioned, but benchmarking Python this way isn't necessarily 
>> representative of how a performance-conscious programmer would reasonably 
>> approach a problem of this kind.
>>
>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[dnm]

Interesting. Did you use the updated Julia code? 

Have you done any comparisons between reading and writing  Numba JIT 
classes and Julia types in tight loops?

On Monday, July 25, 2016 at 10:41:48 AM UTC-4, dexto...@gmail.com wrote:
>
> Just for the sake of comprehensiveness, I ran your Python benchmark 
> through the Numba JIT library (which uses the same underlying LLVM 
> infrastructure that Julia does) and on my computer the Python code is 
> faster than Julia by 68%. Vanilla CPython is terrible for this kind of 
> simple explicit loop code, but Numba and other JIT libraries largely solve 
> that issue with minimal effort as long as the code is simple enough. That 
> by no means solves all of Python's issues in the context of numerical 
> programming and I'm sure the Julia benchmark could be improved as others 
> have already mentioned, but benchmarking Python this way isn't necessarily 
> representative of how a performance-conscious programmer would reasonably 
> approach a problem of this kind.
>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[dextorious]

Just for the sake of comprehensiveness, I ran your Python benchmark through 
the Numba JIT library (which uses the same underlying LLVM infrastructure 
that Julia does) and on my computer the Python code is faster than Julia by 
68%. Vanilla CPython is terrible for this kind of simple explicit loop 
code, but Numba and other JIT libraries largely solve that issue with 
minimal effort as long as the code is simple enough. That by no means 
solves all of Python's issues in the context of numerical programming and 
I'm sure the Julia benchmark could be improved as others have already 
mentioned, but benchmarking Python this way isn't necessarily 
representative of how a performance-conscious programmer would reasonably 
approach a problem of this kind.

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Ferran Mazzanti]

Dear Zhong,

I understand your points very well... as I said I also love and use Julia. 
But at this point I understood that the *only* thing that was discussed was 
speed. And in that matter, a factor 1.6 is still a big difference. In my 
case I'm willing to use it instead of fortran right now, although 1 week 
calculation can turn into 2 weeks, which makes a difference. But for me, 
the speed competition is still
won by other languages. Which are much worse in other fields, I agree :)

Best,

Ferran.

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Jeffrey Sarnoff]

Zong, we would be pleased to increment the tally of Julia users with your 
use.

On Tuesday, July 12, 2016 at 12:00:02 PM UTC-4, Zhong Pan wrote:
>
> Ferran,
>
> I can totally understand Fortran is still the king of fast and efficient 
> numerical calculation in terms of computation time and memory usage. As to 
> "fast", I am comparing Julia main to high-level languages like Python, R, 
> and Matlab which offers similar level of fast prototyping.
>
> Since we are on this topic, I'd like to expand a little. Different users 
> have different needs. As someone trained to be an engineer who can write 
> some code, my desired go-to programming language should have the following:
>
> (1) Fast prototyping: I can get a one-time calculation done or try an idea 
> in very short time - most probably I won't ever repeat it so I don't want 
> to spend much time optimizing it. This requires the language to be simple, 
> expressive, yet flexible and powerful.
>
> (2) Production ready: However, if something I tried is shown to be useful, 
> I can go back and spend some time to improve it such that the code is good 
> for production. "Improve" mainly means making it fast enough, efficient 
> enough, reliable enough, and user friendly enough. Note the "enough" in 
> every criteria - I won't try my best to make it as fast, efficient, 
> reliable, and user friendly as possible, but only enough to satisfy the use 
> case. 
>
> (3) Decent speed for both vectorized calculations and brute-force loops in 
> its native form: Some algos are easier to write and understand in 
> vectorized form, others are easier in loops. I'd like to write an algo in 
> the most natural way without frequently stopping to try fitting a square 
> peg in a round hole. Also, even for fast prototyping, the data can be 
> large, and I don't want to wait for hours or even days to see the result.
>
> (4) Vast collection of free, high-quality libraries/modules: I think 
> everyone wants this. :-)
>
> (5) Open source, most libraries preferably in the same language: Though 
> most of the time I trust other's library, once in a while I may bump into a 
> problem that can only be solved if I can peek under the hood and even tweak 
> a thing or two. It's not just about source code - I want to be able to run 
> my application and trace into the library for debugging, as many subtle 
> bugs can only be revealed when you watch things running.
>
> (6) Easy to collaborate: This requires the language to be simple to pick 
> up - someone can learn enough to start using it in a couple days while 
> learning more advanced features. Code written in it should be easy to read, 
> and most of the time there shouldn't bee too many (obscure) ways to do the 
> same thing. Also it needs to be free: it's hard enough to persuade someone 
> else to learn a new programming language, not to mention persuading anyone 
> to pay hundreds of dollars for using it.
>
> (7) Can scale up to a bigger project with many contributors: The language 
> has enough protections so: (a) Compiler can check for mistakes as much as 
> possible (with some help from the programmer), as run-time errors are 
> annoying and costly; (b) Things can be modularized and collaborators won't 
> easily step on each other's toes (style guide is nice but reinforcement is 
> sometimes necessary).
>
> (8) Cross-platform: Windows and Linux support at the minimum.
>
> (9) Talk to other main-stream languages easily: There will always be a 
> time when one really needs to use a library in C/C++/Fortran/Python/R or 
> even Java/.NET that one desires to use. 
>
> The wishlist is long and demanding. :-) 
>
> For many of the above reasons I was a huge fan of Python. However, Python 
> is way too slow at loops (Cython is not convenient as I often need to 
> access objects and other libraries inside the loops); many errors can only 
> show up in run-time (imagine running an algo for hours and then stop due to 
> one type mismatch - ); and it's scarily easy to change class definition 
> (Programmer A and B can both decide to add a member variable "n" to the 
> object of the same class at run time! And sometimes this is done without 
> the programmer even knowing it, as he/she may think the member variable 
> already exists.); also a lot of powerful Python libraries are written in 
> C/C++, so though they are open source they are effectively "black box" to 
> me unless I am willing to invest a lot of time to dig into the gear box.
>
> So far I see Julia solving several major pain points in Python for me. I'd 
> love to learn about it and use it everyday, and possibly persuade some 
> friends to pick it up, too. :-) 
> My main worry is would Julia is still a very young and somewhat 
> specialized language with limited user base. I hope it will grow to surpass 
> the critical mass for it to become main stream.
>
> Thanks,
> -Zhong
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tuesday, July 12, 2016 at 2:45:33 AM UTC-5, Ferran Mazzanti wrote:
>>
>> 

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

Ferran,

I can totally understand Fortran is still the king of fast and efficient 
numerical calculation in terms of computation time and memory usage. As to 
"fast", I am comparing Julia main to high-level languages like Python, R, 
and Matlab which offers similar level of fast prototyping.

Since we are on this topic, I'd like to expand a little. Different users 
have different needs. As someone trained to be an engineer who can write 
some code, my desired go-to programming language should have the following:

(1) Fast prototyping: I can get a one-time calculation done or try an idea 
in very short time - most probably I won't ever repeat it so I don't want 
to spend much time optimizing it. This requires the language to be simple, 
expressive, yet flexible and powerful.

(2) Production ready: However, if something I tried is shown to be useful, 
I can go back and spend some time to improve it such that the code is good 
for production. "Improve" mainly means making it fast enough, efficient 
enough, reliable enough, and user friendly enough. Note the "enough" in 
every criteria - I won't try my best to make it as fast, efficient, 
reliable, and user friendly as possible, but only enough to satisfy the use 
case. 

(3) Decent speed for both vectorized calculations and brute-force loops in 
its native form: Some algos are easier to write and understand in 
vectorized form, others are easier in loops. I'd like to write an algo in 
the most natural way without frequently stopping to try fitting a square 
peg in a round hole. Also, even for fast prototyping, the data can be 
large, and I don't want to wait for hours or even days to see the result.

(4) Vast collection of free, high-quality libraries/modules: I think 
everyone wants this. :-)

(5) Open source, most libraries preferably in the same language: Though 
most of the time I trust other's library, once in a while I may bump into a 
problem that can only be solved if I can peek under the hood and even tweak 
a thing or two. It's not just about source code - I want to be able to run 
my application and trace into the library for debugging, as many subtle 
bugs can only be revealed when you watch things running.

(6) Easy to collaborate: This requires the language to be simple to pick up 
- someone can learn enough to start using it in a couple days while 
learning more advanced features. Code written in it should be easy to read, 
and most of the time there shouldn't bee too many (obscure) ways to do the 
same thing. Also it needs to be free: it's hard enough to persuade someone 
else to learn a new programming language, not to mention persuading anyone 
to pay hundreds of dollars for using it.

(7) Can scale up to a bigger project with many contributors: The language 
has enough protections so: (a) Compiler can check for mistakes as much as 
possible (with some help from the programmer), as run-time errors are 
annoying and costly; (b) Things can be modularized and collaborators won't 
easily step on each other's toes (style guide is nice but reinforcement is 
sometimes necessary).

(8) Cross-platform: Windows and Linux support at the minimum.

(9) Talk to other main-stream languages easily: There will always be a time 
when one really needs to use a library in C/C++/Fortran/Python/R or even 
Java/.NET that one desires to use. 

The wishlist is long and demanding. :-) 

For many of the above reasons I was a huge fan of Python. However, Python 
is way too slow at loops (Cython is not convenient as I often need to 
access objects and other libraries inside the loops); many errors can only 
show up in run-time (imagine running an algo for hours and then stop due to 
one type mismatch - ); and it's scarily easy to change class definition 
(Programmer A and B can both decide to add a member variable "n" to the 
object of the same class at run time! And sometimes this is done without 
the programmer even knowing it, as he/she may think the member variable 
already exists.); also a lot of powerful Python libraries are written in 
C/C++, so though they are open source they are effectively "black box" to 
me unless I am willing to invest a lot of time to dig into the gear box.

So far I see Julia solving several major pain points in Python for me. I'd 
love to learn about it and use it everyday, and possibly persuade some 
friends to pick it up, too. :-) 
My main worry is would Julia is still a very young and somewhat specialized 
language with limited user base. I hope it will grow to surpass the 
critical mass for it to become main stream.

Thanks,
-Zhong














On Tuesday, July 12, 2016 at 2:45:33 AM UTC-5, Ferran Mazzanti wrote:
>
> ...but then is Julia *really* fast ? I guess that depends on what you 
> compare with. I've been doing lots of numerics for the last 20+ years and 
> for that I use fortran. This is *damn* fast, even more if you use something 
> like the Intel compiler. Fortran many times performs better than C, and the 
> benchmarks 

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Andreas Lobinger]

just my two cents...

On Tuesday, July 12, 2016 at 5:28:24 AM UTC+2, Chris Rackauckas wrote:
>
>
> MATLAB really improved their JIT in 2015b, but as you can see, it cannot 
> come close to Julia. The main reason is that, although it can do quite a 
> bit because it ensures type stability, it has type stability only because 
> everything in MATLAB is by default a complex number. 
>

This is not correct, a variable is by default double. And extended to be 
complex if needed. But that's not the point.
 

> So even in the simplest example (where its JIT works correctly. You'll 
> notice that in many cases it won't work, and since it doesn't cache well it 
> may recompile all the time, and ... it works okay but there's a reason I 
> switched to Julia) you'd expect Julia to be at least twice as fast.
>

I think i have written this before: Julia enables you to write fast code, 
however it's not automatically faster at a constant factor compared to 
something. It's not like: matlab does it wrong, julia does it correct and 
therefore is magically faster. Some people on this mailing list (including 
myself) already had experiences with julia code running surprisingly slow 
or with a heavy memory impact (row-first vs. column-first arrays and 
temporary arrays and similar). 

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Ferran Mazzanti]

btw, I use Julia also, and I really like it very much :)

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

Chris,

Thanks for the comments! I tried @inbounds and @fastmath, which reduced 
Julia execution time from 2.55 sec to 2.46 sec. @simd wouldn't be 
appropriate here as the point is to test brute-force loops that in reality 
are employed in calculations that cannot be vectorized. (And yes, I agree I 
should have used "brute-force". :-))

-Zhong

On Monday, July 11, 2016 at 10:28:24 PM UTC-5, Chris Rackauckas wrote:
>
> You should add @inbounds and try adding @fastmath to the Julia code. Maybe 
> @simd, though the compiler should be doing that automatically. Make sure 
> Julia is compiling with -O3. I wouldn't be surprised if this gets nearly to 
> C++.
>
> If you want to put the random number generation back in, you can improve 
> Julia's time by using ChunkedArrays.jl which allows you to take the random 
> numbers in chunks instead of one at a time in a loop. That should get rid 
> of most of the RNG time in Julia, but the code won't be any more bloated. I 
> think this is one thing to show off in Julia: Julia types (/object) don't 
> have speed penalties, so you can use them to make "fast versions" of just 
> about anything. [Master branch is for Julia v0.5, last tagged version if 
> for Julia v0.4.x. It allows for the random buffer to replenish using a 
> parallel process, but that part had to change in the update]
>
> Other notes:
>
> MATLAB really improved their JIT in 2015b, but as you can see, it cannot 
> come close to Julia. The main reason is that, although it can do quite a 
> bit because it ensures type stability, it has type stability only because 
> everything in MATLAB is by default a complex number. So even in the 
> simplest example (where its JIT works correctly. You'll notice that in many 
> cases it won't work, and since it doesn't cache well it may recompile all 
> the time, and ... it works okay but there's a reason I switched to Julia) 
> you'd expect Julia to be at least twice as fast.
>
> You should test how whether the performance regresses on Julia v0.5 with 
> LLVM 3.7
>
> On Monday, July 11, 2016 at 8:43:12 AM UTC-7, Zhong Pan wrote:
>>
>> Alexander,
>>
>> You are right. I have posted some results in a previous reply with the 
>> calls to random number generator left out. Relatively speaking, C++ 
>> execution time is reduced more significantly than others, making it the 
>> fastest in that test. 
>>
>> -Zhong
>>
>> On Monday, July 11, 2016 at 9:25:08 AM UTC-5, Alexander Ranaldi wrote:
>>>
>>> I am not sure calling the rand function in a loop is fair; you're 
>>> benchmarking that function, not the mathematics.  Perhaps allocate a vector 
>>> of random numbers and index them rather than calling rand repeatedly.
>>>
>>>
>>>
>>> On Monday, July 11, 2016 at 8:09:28 AM UTC-4, David Barton wrote:

 For reference, with Matlab 2016a: 4.97 sec; Julia 0.4.6: 2.76 sec; 
 Python 3.5.1: 166.76 sec.

 Note that there is a mistake in your Matlab code - zeros(n) returns an 
 n by n matrix of zeros (hence running out of memory). Instead you want 
 zeros(1, n) to get a vector.

 David

 On Monday, 11 July 2016 10:07:01 UTC+1, Zhong Pan wrote:
>
> Hi Andreas,
>
> Thanks for the comments.
>
> * If someone has a more recent Matlab it'll be interesting to try. The 
> license is so expensive and I don't have access to newer version now.
>
> * Yes you are right, I also realized that I don't know how much the 
> random number generator implementation difference would contribute. One 
> thing to try is to leave out the random number generations. 
>
> I tried it and here's the result: Python 166.44 sec (107.4x, was 
> 64.3x), Julia 2.56 sec (1.7x, was 0.8x), VC++ 1.55 sec (1.0x as 
> reference), 
> C#.NET 3.49 sec (2.3x, was 1.1x), Java 10.14 sec (6.5x, was 3.0x), and 
> Matlab 7.75 sec (5.0x, was 3.3x). Therefore, it seems VC++ improved the 
> most by removing random number generations, and other languages just all 
> look relatively 1.5 to 2.2 times more slower. Julia is still the fastest 
> aside from VC++, and C#.NET is still not far behind.
>
> Cheers,
> -Zhong
>
>
> On Monday, July 11, 2016 at 3:27:30 AM UTC-5, Andreas Lobinger wrote:
>>
>> 2 small things:
>>
>> * a more recent Matlab should already be faster, especially in this 
>> loop thing
>> * random generators' runtime -depending on the complexity they spend- 
>> really makes a difference.
>>
>>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Chris Rackauckas]

Also: it's "brute force", not "brutal-force". The connotation is quite 
different :). 

On Monday, July 11, 2016 at 9:21:34 AM UTC-7, Zhong Pan wrote:
>
> Thanks to all the helpful comments. Just for the sake of tidiness, I 
> attached rev 3 of the (now even simpler) benchmark result PDF.
>
> Changes are:
> * Removed random number generation as it was pointed out that random 
> generators are relatively complex, and difference in implementation can 
> lead to unfairly large impact on the test results.
> * Fixed the bug in Matlab code that had caused memory overflow. Now codes 
> in all languages run the same number of loops for same size arrays.
> * I found if I wrap the Python code in a function, too, the performance is 
> also improved, although just slightly and far less dramatic comparing to 
> what happened to me in Julia.
>
> I think I got what I am looking for in this simple test. Look forward to 
> using Julia a lot and enjoying the experience.
>
> -Zhong
>
>
>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Chris Rackauckas]

You should add @inbounds and try adding @fastmath to the Julia code. Maybe 
@simd, though the compiler should be doing that automatically. Make sure 
Julia is compiling with -O3. I wouldn't be surprised if this gets nearly to 
C++.

If you want to put the random number generation back in, you can improve 
Julia's time by using ChunkedArrays.jl which allows you to take the random 
numbers in chunks instead of one at a time in a loop. That should get rid 
of most of the RNG time in Julia, but the code won't be any more bloated. I 
think this is one thing to show off in Julia: Julia types (/object) don't 
have speed penalties, so you can use them to make "fast versions" of just 
about anything. [Master branch is for Julia v0.5, last tagged version if 
for Julia v0.4.x. It allows for the random buffer to replenish using a 
parallel process, but that part had to change in the update]

Other notes:

MATLAB really improved their JIT in 2015b, but as you can see, it cannot 
come close to Julia. The main reason is that, although it can do quite a 
bit because it ensures type stability, it has type stability only because 
everything in MATLAB is by default a complex number. So even in the 
simplest example (where its JIT works correctly. You'll notice that in many 
cases it won't work, and since it doesn't cache well it may recompile all 
the time, and ... it works okay but there's a reason I switched to Julia) 
you'd expect Julia to be at least twice as fast.

You should test how whether the performance regresses on Julia v0.5 with 
LLVM 3.7

On Monday, July 11, 2016 at 8:43:12 AM UTC-7, Zhong Pan wrote:
>
> Alexander,
>
> You are right. I have posted some results in a previous reply with the 
> calls to random number generator left out. Relatively speaking, C++ 
> execution time is reduced more significantly than others, making it the 
> fastest in that test. 
>
> -Zhong
>
> On Monday, July 11, 2016 at 9:25:08 AM UTC-5, Alexander Ranaldi wrote:
>>
>> I am not sure calling the rand function in a loop is fair; you're 
>> benchmarking that function, not the mathematics.  Perhaps allocate a vector 
>> of random numbers and index them rather than calling rand repeatedly.
>>
>>
>>
>> On Monday, July 11, 2016 at 8:09:28 AM UTC-4, David Barton wrote:
>>>
>>> For reference, with Matlab 2016a: 4.97 sec; Julia 0.4.6: 2.76 sec; 
>>> Python 3.5.1: 166.76 sec.
>>>
>>> Note that there is a mistake in your Matlab code - zeros(n) returns an n 
>>> by n matrix of zeros (hence running out of memory). Instead you want 
>>> zeros(1, n) to get a vector.
>>>
>>> David
>>>
>>> On Monday, 11 July 2016 10:07:01 UTC+1, Zhong Pan wrote:

 Hi Andreas,

 Thanks for the comments.

 * If someone has a more recent Matlab it'll be interesting to try. The 
 license is so expensive and I don't have access to newer version now.

 * Yes you are right, I also realized that I don't know how much the 
 random number generator implementation difference would contribute. One 
 thing to try is to leave out the random number generations. 

 I tried it and here's the result: Python 166.44 sec (107.4x, was 
 64.3x), Julia 2.56 sec (1.7x, was 0.8x), VC++ 1.55 sec (1.0x as 
 reference), 
 C#.NET 3.49 sec (2.3x, was 1.1x), Java 10.14 sec (6.5x, was 3.0x), and 
 Matlab 7.75 sec (5.0x, was 3.3x). Therefore, it seems VC++ improved the 
 most by removing random number generations, and other languages just all 
 look relatively 1.5 to 2.2 times more slower. Julia is still the fastest 
 aside from VC++, and C#.NET is still not far behind.

 Cheers,
 -Zhong


 On Monday, July 11, 2016 at 3:27:30 AM UTC-5, Andreas Lobinger wrote:
>
> 2 small things:
>
> * a more recent Matlab should already be faster, especially in this 
> loop thing
> * random generators' runtime -depending on the complexity they spend- 
> really makes a difference.
>
>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Andreas Lobinger]

On Monday, July 11, 2016 at 6:38:13 PM UTC+2, Sisyphuss wrote:
>
> It's surprising to see Python so slow and Matlab so fast.
>
>  Matlab: does the JIT compiler translate the loops to C?
>

Not to C, rather directly to machine code. LLVM seems to be in use here, 
also.
 

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Sisyphuss]

It's surprising to see Python so slow and Matlab so fast.

 Python: maybe it will get much faster when using comprehension?
 Matlab: does the JIT compiler translate the loops to C?


On Monday, July 11, 2016 at 2:09:28 PM UTC+2, David Barton wrote:
>
> For reference, with Matlab 2016a: 4.97 sec; Julia 0.4.6: 2.76 sec; Python 
> 3.5.1: 166.76 sec.
>
> Note that there is a mistake in your Matlab code - zeros(n) returns an n 
> by n matrix of zeros (hence running out of memory). Instead you want 
> zeros(1, n) to get a vector.
>
> David
>
> On Monday, 11 July 2016 10:07:01 UTC+1, Zhong Pan wrote:
>>
>> Hi Andreas,
>>
>> Thanks for the comments.
>>
>> * If someone has a more recent Matlab it'll be interesting to try. The 
>> license is so expensive and I don't have access to newer version now.
>>
>> * Yes you are right, I also realized that I don't know how much the 
>> random number generator implementation difference would contribute. One 
>> thing to try is to leave out the random number generations. 
>>
>> I tried it and here's the result: Python 166.44 sec (107.4x, was 64.3x), 
>> Julia 2.56 sec (1.7x, was 0.8x), VC++ 1.55 sec (1.0x as reference), C#.NET 
>> 3.49 sec (2.3x, was 1.1x), Java 10.14 sec (6.5x, was 3.0x), and Matlab 7.75 
>> sec (5.0x, was 3.3x). Therefore, it seems VC++ improved the most by 
>> removing random number generations, and other languages just all look 
>> relatively 1.5 to 2.2 times more slower. Julia is still the fastest aside 
>> from VC++, and C#.NET is still not far behind.
>>
>> Cheers,
>> -Zhong
>>
>>
>> On Monday, July 11, 2016 at 3:27:30 AM UTC-5, Andreas Lobinger wrote:
>>>
>>> 2 small things:
>>>
>>> * a more recent Matlab should already be faster, especially in this loop 
>>> thing
>>> * random generators' runtime -depending on the complexity they spend- 
>>> really makes a difference.
>>>
>>>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

Thanks to all the helpful comments. Just for the sake of tidiness, I 
attached rev 3 of the (now even simpler) benchmark result PDF.

Changes are:
* Removed random number generation as it was pointed out that random 
generators are relatively complex, and difference in implementation can 
lead to unfairly large impact on the test results.
* Fixed the bug in Matlab code that had caused memory overflow. Now codes 
in all languages run the same number of loops for same size arrays.
* I found if I wrap the Python code in a function, too, the performance is 
also improved, although just slightly and far less dramatic comparing to 
what happened to me in Julia.

I think I got what I am looking for in this simple test. Look forward to 
using Julia a lot and enjoying the experience.

-Zhong




Simple_Benchmark_For_Brutal-force_Loops_rev3.pdf
Description: Adobe PDF document

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

David,

Thanks for the test results and the correction - now I recall how it's done 
in Matlab. Haven't been using it for a while. :-)

-Zhong

On Monday, July 11, 2016 at 7:09:28 AM UTC-5, David Barton wrote:
>
> For reference, with Matlab 2016a: 4.97 sec; Julia 0.4.6: 2.76 sec; Python 
> 3.5.1: 166.76 sec.
>
> Note that there is a mistake in your Matlab code - zeros(n) returns an n 
> by n matrix of zeros (hence running out of memory). Instead you want 
> zeros(1, n) to get a vector.
>
> David
>
> On Monday, 11 July 2016 10:07:01 UTC+1, Zhong Pan wrote:
>>
>> Hi Andreas,
>>
>> Thanks for the comments.
>>
>> * If someone has a more recent Matlab it'll be interesting to try. The 
>> license is so expensive and I don't have access to newer version now.
>>
>> * Yes you are right, I also realized that I don't know how much the 
>> random number generator implementation difference would contribute. One 
>> thing to try is to leave out the random number generations. 
>>
>> I tried it and here's the result: Python 166.44 sec (107.4x, was 64.3x), 
>> Julia 2.56 sec (1.7x, was 0.8x), VC++ 1.55 sec (1.0x as reference), C#.NET 
>> 3.49 sec (2.3x, was 1.1x), Java 10.14 sec (6.5x, was 3.0x), and Matlab 7.75 
>> sec (5.0x, was 3.3x). Therefore, it seems VC++ improved the most by 
>> removing random number generations, and other languages just all look 
>> relatively 1.5 to 2.2 times more slower. Julia is still the fastest aside 
>> from VC++, and C#.NET is still not far behind.
>>
>> Cheers,
>> -Zhong
>>
>>
>> On Monday, July 11, 2016 at 3:27:30 AM UTC-5, Andreas Lobinger wrote:
>>>
>>> 2 small things:
>>>
>>> * a more recent Matlab should already be faster, especially in this loop 
>>> thing
>>> * random generators' runtime -depending on the complexity they spend- 
>>> really makes a difference.
>>>
>>>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[David Barton]

For reference, with Matlab 2016a: 4.97 sec; Julia 0.4.6: 2.76 sec; Python 
3.5.1: 166.76 sec.

Note that there is a mistake in your Matlab code - zeros(n) returns an n by 
n matrix of zeros (hence running out of memory). Instead you want zeros(1, 
n) to get a vector.

David

On Monday, 11 July 2016 10:07:01 UTC+1, Zhong Pan wrote:
>
> Hi Andreas,
>
> Thanks for the comments.
>
> * If someone has a more recent Matlab it'll be interesting to try. The 
> license is so expensive and I don't have access to newer version now.
>
> * Yes you are right, I also realized that I don't know how much the random 
> number generator implementation difference would contribute. One thing to 
> try is to leave out the random number generations. 
>
> I tried it and here's the result: Python 166.44 sec (107.4x, was 64.3x), 
> Julia 2.56 sec (1.7x, was 0.8x), VC++ 1.55 sec (1.0x as reference), C#.NET 
> 3.49 sec (2.3x, was 1.1x), Java 10.14 sec (6.5x, was 3.0x), and Matlab 7.75 
> sec (5.0x, was 3.3x). Therefore, it seems VC++ improved the most by 
> removing random number generations, and other languages just all look 
> relatively 1.5 to 2.2 times more slower. Julia is still the fastest aside 
> from VC++, and C#.NET is still not far behind.
>
> Cheers,
> -Zhong
>
>
> On Monday, July 11, 2016 at 3:27:30 AM UTC-5, Andreas Lobinger wrote:
>>
>> 2 small things:
>>
>> * a more recent Matlab should already be faster, especially in this loop 
>> thing
>> * random generators' runtime -depending on the complexity they spend- 
>> really makes a difference.
>>
>>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Zhong Pan]

Hi Andreas,

Thanks for the comments.

* If someone has a more recent Matlab it'll be interesting to try. The 
license is so expensive and I don't have access to newer version now.

* Yes you are right, I also realized that I don't know how much the random 
number generator implementation difference would contribute. One thing to 
try is to leave out the random number generations. 

I tried it and here's the result: Python 166.44 sec (107.4x, was 64.3x), 
Julia 2.56 sec (1.7x, was 0.8x), VC++ 1.55 sec (1.0x as reference), C#.NET 
3.49 sec (2.3x, was 1.1x), Java 10.14 sec (6.5x, was 3.0x), and Matlab 7.75 
sec (5.0x, was 3.3x). Therefore, it seems VC++ improved the most by 
removing random number generations, and other languages just all look 
relatively 1.5 to 2.2 times more slower. Julia is still the fastest aside 
from VC++, and C#.NET is still not far behind.

Cheers,
-Zhong


On Monday, July 11, 2016 at 3:27:30 AM UTC-5, Andreas Lobinger wrote:
>
> 2 small things:
>
> * a more recent Matlab should already be faster, especially in this loop 
> thing
> * random generators' runtime -depending on the complexity they spend- 
> really makes a difference.
>
>

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Andreas Lobinger]

2 small things:

* a more recent Matlab should already be faster, especially in this loop 
thing
* random generators' runtime -depending on the complexity they spend- 
really makes a difference.

[julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast!

[Mosè Giordano]

Hi Zhong,

you may want to check out Julia 0.5, on my box your benchmark is ~13% 
faster with Julia 0.5: 3.367658 seconds with Julia 0.4.6 and 2.898068 
seconds with Julia 0.5.


Bye,
Mosè


Hi,
>
> Sorry I have to post a revision so quickly. Just after I posted my 
> previous benchmark, I found the message on the top advising against using 
> global variables when running performance test. I realized I did exactly 
> that, so I went back to redo my test wrapped in a function. And that did 
> the magic! The difference is a ~18x speedup!
>
> I didn't know how to recall my previous post, so I have to post a revised 
> one (attached PDF) here. Since there is no surprise finding about Julia 
> anymore, the benchmark results are not that interesting now; but I have to 
> make sure the information I posted is not misleading, so here you go.
>
> Cheers,
> -Zhong
>
>