Re: [ClojureScript] Re: How does Clojurescript compilation scales with CPU?

2020-01-17 Thread Khalid Jebbari 
I may do some testing in the a close future and report here. I will either
do it in our project or in a an open-source project that's "relevant" to
show real-world numbers. Do you have an idea of such project? I think I
will try to bench compilation time in advanced mode with a different number
of cores activated by BIOS (I will soon have a 16C/32T machine, I hope the
BIOS will allow me to deactivate some of them).
Khalid aka DjebbZ
@Dj3bbZ


On Fri, Jan 17, 2020 at 11:58 AM Thomas Heller  wrote:

> Depends on the size I'd say but in theory yes. Lots of things factor into
> the compilation times, even tiny namespaces can take a long time to compile
> if some macro just takes a long time to do its thing.
>
> On Friday, January 17, 2020 at 11:18:21 AM UTC+1, Khalid Jebbari wrote:
>>
>> Thanks for the detailed answer. Does it somewhat mean that splitting code
>> into smaller namespaces can achieve faster compilation thanks to
>> parallelization ?
>>
>>
>> On Fri, Jan 17, 2020 at 10:43 AM Thomas Heller  wrote:
>>
>>> It depends on the namespaces used. In general a CLJS namespaces can only
>>> be compiled once all its dependencies have been compiled. So if those
>>> dependendencies can be compiled in parallel they will use multiple threads
>>> from a pool, which should keep all cores busy. In my experience a good
>>> balance between core count and core speed matters. If you have big
>>> namespaces that a lot of other namespaces depend on (eg. like cljs.core)
>>> then its compilation will "block" all other threads so its important it
>>> finishes fast (ie. fast cores). If you have lots a small namespaces that
>>> are mostly independent then you can get maximum parallelization (ie. many
>>> cores).
>>>
>>> I have a i7-8700K 6c and there are builds that aren't able to use all
>>> cores due to the namespace setup (few very large ones). Others happily use
>>> everything. Single core difference is gigantic to my previous CPU from a
>>> macbook pro 2016.
>>>
>>> If you really really want to torture your CPU you can try
>>> https://github.com/mfikes/fifth-postulate or
>>> https://github.com/mfikes/coal-mine to compare.
>>>
>>> HTH,
>>> Thomas
>>>
>>> On Friday, January 17, 2020 at 5:29:06 AM UTC+1, Khalid Jebbari wrote:

 Hello,

 We're using the parallel build option, and I noticed the difference in
 speed between my laptop and other laptops is basically proportional to the
 difference in speed of the CPUs (based on notebookcheck's benchmarks). I
 have 4C/8T 7700HQ CPU and my colleagues have a 8565U iirc (some have the
 6600U). Mine is almost twice as fast in benchmarks, which is reflected in
 cljs compilation times.

 So my question is how does the cljs compiler scale with regards to CPU?
 Core count? Single thread perf? All cores frequencies? Is it capped to some
 number of cores?

 --
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "ClojureScript" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/clojurescript/PwpVJNrF0Zc/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> clojur...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/clojurescript/7a967f70-995e-49d3-9b25-a5b327689736%40googlegroups.com
>>> 
>>> .
>>>
>> --
> Note that posts from new members are moderated - please be patient with
> your first post.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "ClojureScript" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojurescript/PwpVJNrF0Zc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojurescript+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/clojurescript/fe80b84a-4c23-4789-9b27-8f84c631d93d%40googlegroups.com
> 
> .
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojurescript/CAM3R3BCE7cL9cBrPAVMKnzd6SZ65p3XeXbMM0wFMhs%2BjvqxN1A%40mail.gmail.com.

Re: [ClojureScript] Re: How does Clojurescript compilation scales with CPU?

2020-01-17 Thread Thomas Heller 
Depends on the size I'd say but in theory yes. Lots of things factor into 
the compilation times, even tiny namespaces can take a long time to compile 
if some macro just takes a long time to do its thing.

On Friday, January 17, 2020 at 11:18:21 AM UTC+1, Khalid Jebbari wrote:
>
> Thanks for the detailed answer. Does it somewhat mean that splitting code 
> into smaller namespaces can achieve faster compilation thanks to 
> parallelization ?
>
>
> On Fri, Jan 17, 2020 at 10:43 AM Thomas Heller  > wrote:
>
>> It depends on the namespaces used. In general a CLJS namespaces can only 
>> be compiled once all its dependencies have been compiled. So if those 
>> dependendencies can be compiled in parallel they will use multiple threads 
>> from a pool, which should keep all cores busy. In my experience a good 
>> balance between core count and core speed matters. If you have big 
>> namespaces that a lot of other namespaces depend on (eg. like cljs.core) 
>> then its compilation will "block" all other threads so its important it 
>> finishes fast (ie. fast cores). If you have lots a small namespaces that 
>> are mostly independent then you can get maximum parallelization (ie. many 
>> cores). 
>>
>> I have a i7-8700K 6c and there are builds that aren't able to use all 
>> cores due to the namespace setup (few very large ones). Others happily use 
>> everything. Single core difference is gigantic to my previous CPU from a 
>> macbook pro 2016.
>>
>> If you really really want to torture your CPU you can try 
>> https://github.com/mfikes/fifth-postulate or 
>> https://github.com/mfikes/coal-mine to compare.
>>
>> HTH,
>> Thomas
>>
>> On Friday, January 17, 2020 at 5:29:06 AM UTC+1, Khalid Jebbari wrote:
>>>
>>> Hello,
>>>
>>> We're using the parallel build option, and I noticed the difference in 
>>> speed between my laptop and other laptops is basically proportional to the 
>>> difference in speed of the CPUs (based on notebookcheck's benchmarks). I 
>>> have 4C/8T 7700HQ CPU and my colleagues have a 8565U iirc (some have the 
>>> 6600U). Mine is almost twice as fast in benchmarks, which is reflected in 
>>> cljs compilation times.
>>>
>>> So my question is how does the cljs compiler scale with regards to CPU? 
>>> Core count? Single thread perf? All cores frequencies? Is it capped to some 
>>> number of cores?
>>>
>>> -- 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> --- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "ClojureScript" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/clojurescript/PwpVJNrF0Zc/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> clojur...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/clojurescript/7a967f70-995e-49d3-9b25-a5b327689736%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojurescript/fe80b84a-4c23-4789-9b27-8f84c631d93d%40googlegroups.com.

Re: [ClojureScript] Re: How does Clojurescript compilation scales with CPU?

2020-01-17 Thread Khalid Jebbari 
Thanks for the detailed answer. Does it somewhat mean that splitting code
into smaller namespaces can achieve faster compilation thanks to
parallelization ?


On Fri, Jan 17, 2020 at 10:43 AM Thomas Heller  wrote:

> It depends on the namespaces used. In general a CLJS namespaces can only
> be compiled once all its dependencies have been compiled. So if those
> dependendencies can be compiled in parallel they will use multiple threads
> from a pool, which should keep all cores busy. In my experience a good
> balance between core count and core speed matters. If you have big
> namespaces that a lot of other namespaces depend on (eg. like cljs.core)
> then its compilation will "block" all other threads so its important it
> finishes fast (ie. fast cores). If you have lots a small namespaces that
> are mostly independent then you can get maximum parallelization (ie. many
> cores).
>
> I have a i7-8700K 6c and there are builds that aren't able to use all
> cores due to the namespace setup (few very large ones). Others happily use
> everything. Single core difference is gigantic to my previous CPU from a
> macbook pro 2016.
>
> If you really really want to torture your CPU you can try
> https://github.com/mfikes/fifth-postulate or
> https://github.com/mfikes/coal-mine to compare.
>
> HTH,
> Thomas
>
> On Friday, January 17, 2020 at 5:29:06 AM UTC+1, Khalid Jebbari wrote:
>>
>> Hello,
>>
>> We're using the parallel build option, and I noticed the difference in
>> speed between my laptop and other laptops is basically proportional to the
>> difference in speed of the CPUs (based on notebookcheck's benchmarks). I
>> have 4C/8T 7700HQ CPU and my colleagues have a 8565U iirc (some have the
>> 6600U). Mine is almost twice as fast in benchmarks, which is reflected in
>> cljs compilation times.
>>
>> So my question is how does the cljs compiler scale with regards to CPU?
>> Core count? Single thread perf? All cores frequencies? Is it capped to some
>> number of cores?
>>
>> --
> Note that posts from new members are moderated - please be patient with
> your first post.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "ClojureScript" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojurescript/PwpVJNrF0Zc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojurescript+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/clojurescript/7a967f70-995e-49d3-9b25-a5b327689736%40googlegroups.com
> 
> .
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojurescript/CAM3R3BARm8m08%3DA1aSN%2B3rW9E0DUNGyuzyWgGTmf5fscGnFjpA%40mail.gmail.com.

[ClojureScript] Re: How does Clojurescript compilation scales with CPU?

2020-01-17 Thread Thomas Heller 
It depends on the namespaces used. In general a CLJS namespaces can only be 
compiled once all its dependencies have been compiled. So if those 
dependendencies can be compiled in parallel they will use multiple threads 
from a pool, which should keep all cores busy. In my experience a good 
balance between core count and core speed matters. If you have big 
namespaces that a lot of other namespaces depend on (eg. like cljs.core) 
then its compilation will "block" all other threads so its important it 
finishes fast (ie. fast cores). If you have lots a small namespaces that 
are mostly independent then you can get maximum parallelization (ie. many 
cores). 

I have a i7-8700K 6c and there are builds that aren't able to use all cores 
due to the namespace setup (few very large ones). Others happily use 
everything. Single core difference is gigantic to my previous CPU from a 
macbook pro 2016.

If you really really want to torture your CPU you can try 
https://github.com/mfikes/fifth-postulate or 
https://github.com/mfikes/coal-mine to compare.

HTH,
Thomas

On Friday, January 17, 2020 at 5:29:06 AM UTC+1, Khalid Jebbari wrote:
>
> Hello,
>
> We're using the parallel build option, and I noticed the difference in 
> speed between my laptop and other laptops is basically proportional to the 
> difference in speed of the CPUs (based on notebookcheck's benchmarks). I 
> have 4C/8T 7700HQ CPU and my colleagues have a 8565U iirc (some have the 
> 6600U). Mine is almost twice as fast in benchmarks, which is reflected in 
> cljs compilation times.
>
> So my question is how does the cljs compiler scale with regards to CPU? 
> Core count? Single thread perf? All cores frequencies? Is it capped to some 
> number of cores?
>
>

-- 
Note that posts from new members are moderated - please be patient with your 
first post.
--- 
You received this message because you are subscribed to the Google Groups 
"ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojurescript+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojurescript/7a967f70-995e-49d3-9b25-a5b327689736%40googlegroups.com.