On Fri, Dec 23, 2022 at 5:28 PM Henry Rich wrote:
> J threading shines when:
>
> * you want the threads to share data
>
> * you don't know whether threading will help you or not, and you'd like
> to see. It's so easy to make a J verb into a task.
Also when you have a task where threading is an
If the alternative is to create a new process for each application, that
is a form of multithreading and you wouldn't expect that threading in J
would be much faster.
J threading shines when:
* you want the threads to share data
* you don't know whether threading will help you or not, and
Yes - it takes about the same time as the older method where I
pre-allocated the files and created a separate process for each
allocation. However, the current run-time is much less variable than the
older one and slightly faster. I run it a few times a week and it's been
fine for months now.
Hi Devon,
I hope you are well.
About your photo flipping exercise.
Is it now running stably? (at one point you mentioned that it begab failing)
How are files distributed among folders?
Approximately how many files and folders are you running your test across?
Are you always using the same data?
0 is the threadpool in your example, always.
'worker' means the task will always be executed in a worker thread that
is NOT the thread that started the task (usually the task is started in
the master thread).
Henry Rich
On 12/22/2022 2:50 PM, Ak O wrote:
On Wed., Dec. 21, 2022, 19:20 bill
On Wed., Dec. 21, 2022, 19:20 bill lam, wrote:
> create thread pool:
> MAXTHREADS_z_=: {: 8 T.''
> echo 'MAXTHREADS: ',": MAXTHREADS
> 3 : 0''
> if. 1 0"0^:(0 < #) 0$~ (1 >. (<:9!:56'cores') <. (<:MAXTHREADS)- 1 T. '')
> echo 'thread pool#0: ', ": n1=. 1 T. ''
> 0"0^:(0 < #) 1$~ (1 >.
As a contrived example, to find the maximum within 10 sets of numbers
>./@:> t.''"0[ _1e5<\ ?1e6#1e10
+--+--+--+--+--+--+--+--+--+--+
I am not sure. But you can check the number of thread created.
On Thu, 22 Dec 2022 at 2:28 PM Ak O wrote:
> Thank you Bill,
>
> Are these two lines equivalent?
>
> T.&''"0 (|change)#(*change){1 0 55
>
> 0''0^:(0 < #)1$~(1>. (<:MAXTHREADS)-1 T.'')
>
>
> Raul's example:
>
> setthreadcount=: {{
>
Thank you Bill,
Are these two lines equivalent?
T.&''"0 (|change)#(*change){1 0 55
0''0^:(0 < #)1$~(1>. (<:MAXTHREADS)-1 T.'')
Raul's example:
setthreadcount=: {{
change=. y-1 T.''
T.&''"0 (|change)#(*change){0 0 55
1 T.''
}}
Ak.
On Wed., Dec. 21, 2022, 19:20 bill
Oh, wait.. I guess I'm ignorant of the thread grouping issues.
Please ignore this previous email.
Thanks,
--
Raul
On Wed, Dec 21, 2022 at 10:55 PM Raul Miller wrote:
>
> Note that you could have used the 'setthreadcount' implementation for this:
>
> setthreadcount <:{:8T.''
>
> However, I'm
Note that you could have used the 'setthreadcount' implementation for this:
setthreadcount <:{:8T.''
However, I'm not sure that creating the maximum number of threads is
going to be the most efficient approach. (In some cases it might be.)
Also, I'm realizing that I should have included that
On Wed, Dec 21, 2022 at 8:07 PM Ak O wrote:
> setthreadcount=: {{
> change=. y-1 T.''
> T.&''"0 (|change)#(*change){0 0 55
> 1 T.''
> }}
>
> To accomplish this, you might use a routine like
> setthreadcount=: {{
>change=. y-1 T.''
> -NB. Does this set 'change' to a
create thread pool:
MAXTHREADS_z_=: {: 8 T.''
echo 'MAXTHREADS: ',": MAXTHREADS
3 : 0''
if. 1. (<:9!:56'cores') <. (<:MAXTHREADS)- 1 T. '')
echo 'thread pool#0: ', ": n1=. 1 T. ''
0"0^:(0 < #) 1$~ (1 >. (<:MAXTHREADS)- 1 T. '')
echo 'thread pool#1: ', ": (1 T. '') - n1
end.
)
for cpu intensive
Hi Bill,
How is a threadpool assigned?
For clarity, can you please show two sample commands?
The command that creates threads in threadpool 0.
The command that creates threads in threadpool 1.
Thank you for your help.
Ak
>
Hi Ed,
I hope you are well.
If you are willing, do you mind sharing a comparison of the routine that
you improved (with some commentary)?
Ak
>
>
--
For information about J forums see http://www.jsoftware.com/forums.htm
It would be sufficient to create threadpool when starting an application.
No need to bother how many threads are available inside the application.
Eg if your machine has 8 CPU, then create 7 threads for pool 0 (for CPU
bounded tasks) and 56 threads for pool 1 (for I/O bounded tasks)
On Thu, Dec
Thank you Raul,
Conceptually, the primitives are simple. T. is for configuration and
setup. You always have one thread (the master thread), and you
probably should have one additional thread for each cpu "core" (which
you might think of as a register set -- a small set of variables that
L2 can usually be expected to be between 256k-1m. (Threads can still be
helpful even once you run out of l2; benchmark and see. Apple arm is a bit
weird here--though je is mostly optimised for intel anyway.)
On Wed, 21 Dec 2022, Raul Miller wrote:
On Tue, Dec 20, 2022 at 6:21 PM Henry Rich
On Wed, 21 Dec 2022, Raul Miller wrote:
If there's a way of checking cache sizes in C, it might be nice to have a
9!: foreign or a T. query to report this kind of architectural information.
It is on the list. But turns out to be surprisingly subtle/annoying.
On Tue, Dec 20, 2022 at 6:21 PM Henry Rich wrote:
> The ideal candidate for a task is a compute-intensive
> function that doesn't need much more memory than a core's L2 cache.
On linux you can find the size of L2 cache from the os command line:
lscpu | grep L2
On windows you can find the
You can do whatever you like in parallel. All tasks share the same J
public namespaces, so you have to make sure they don't interfere with
each other. The ideal candidate for a task is a compute-intensive
function that doesn't need much more memory than a core's L2 cache.
Raul's message
Raul,
I will offer one narrow use case where threading shines. I used the T
primitives recently with gethttp calls to speed a web crawling routine by a
factor of 60, which brought the problem from infeasible to near-trivial. So
while I agree that performance gains on CPU-bound tasks may be
Hi Henry,
I have seen the Glossary entries. I agree with you that understanding the
meaning of the words will go a long way. I might not be expressing myself
clearly.
Please bear with me.
In the context of threading what are these things?
_
'keyword', Do I assign it?
Is it always 'worker'?
Knowing where to put the primitives is indeed the issue. That's why
they're primitives, basically.
Conceptually, the primitives are simple. T. is for configuration and
setup. You always have one thread (the master thread), and you
probably should have one additional thread for each cpu "core"
Hi Devon,
Thank you for the iterations of your exercise. I have studied these posts
along with your earliest proto examples in the multi-threading forum posts.
My challenge is understanding which Primitives go where in the context of
building up a threading routine for a directed task.
Your questions 'what is...' are answered by
https://code.jsoftware.com/wiki/Vocabulary/Glossary and if any are
missing, report that fact and I'll add them. I think once you
understand what the words mean most of your questions will be answered;
please re-ask anything that is still unclear.
Hi Ak,
That's a lot of questions. I, too, would like to better understand pyxs.
In the meantime, here are some things I've written up on multithreading:
https://code.jsoftware.com/wiki/NYCJUG/2022-05-10#Trying_to_Multithread,
I hope you are all well. Thank you for your hard work on this subject.
I am having difficulty understanding how to correctly apply the T. and t.
(tcapdot/tdot) Primitives based on the vocabulary documentation pages.
The concept of threading is new to me. The documentation pages (T./t.) use
many
28 matches
Mail list logo
