Clarification of my statement that arguments are not copied: not
explicitly copied. If they are not in cache in the thread that runs on
them, they will be copied from D3$ (a slow copy).
Henry Rich
On 5/25/2022 3:46 PM, Jan-Pieter Jacobs wrote:
Very interesting insights indeed!
@Pascal: indeed, I wasn't thinking of parallelising such simple verbs, it
is clear that the more work is needed, the less overhead is going to be
present. Although apparently, verbs don't have to be that complicated to
benefit (at least a little):
NB. both outer tines parallel
pfr=: {{ (t.'')(`(u&:>))(`(v t.''))(`:6)}}
NB. only right tine parallel, suggested by Elijah
pf=: {{ (`(u >))(`(v t.''))(`:6)}}
0&T.@''^:8''
8
in=: i. 1e6
1000 timespacex '(+/ % pfr (*/)) in'
0.00289912 9856
1000 timespacex '(+/ % pf (*/)) in'
0.00263188 8000
1000 timespacex '(+/ % (*/)) in' NB. no tasks
0.0029901 2240
Indeed, spinning up a task only for the right tine is faster, and I can't
imagine any case that that would be slower than doing both.
Jan-Pieter
On Wed, 25 May 2022, 13:57 Henry Rich, <[email protected]> wrote:
Arguments to a thread are not copied. If they are virtual, they are
realized.
Elijah is right in his analysis but I do not share his view that virtual
arguments need not be realized.
Henry Rich
On 5/25/2022 12:20 AM, 'Pascal Jasmin' via Beta wrote:
This can be done tacitly with modifier trains
v2c =: 1 : '[. u ].'
Ft =: {{(t.'') u&> v2c (t. '')}}
Ftl =: {{]: u&> v2c (t. '')}}
+/ % Ftl #
+/ %&> #t.''
+/ % Ft #
+/t.'' %&> #t.''
But, it seems like forks need to be tailored to domain, unless we'd
escape some operations. There is a high "copy data" overhead to starting a
thread. So, with t =: t. ''
@] t and @]~ t would always benefit from t @] or t@]~
monads # $ [ ] , ; ,. ,: {. {: |.|:. and potentially others should
probably never run on a thread due to instant result.
Or if form u@v@v t for any/all v's in above list, expression should be
transformed as u t @v@v
On Monday, May 23, 2022, 09:48:47 p.m. EDT, Elijah Stone <
[email protected]> wrote:
I suggest using f (g >) h t.'' instead of f t.'' g&:> h t.'', as it is
somewhat faster.
On Sun, 22 May 2022, Jan-Pieter Jacobs wrote:
I've had some fun with parallel primitives and forks, and came up with
two
approaches that result in the exact same result, but have different
sytax.
The first converts to AR and replaces the verbs by applying (t.'') and
(&>)
to the outer and inner tines of the fork, respectively:
NB. parallel single fork
psf =: {{
'cod dat' =. > u toar
if. '3'=cod do. NB. if fork
dat =. pf dat
end.
(< cod ,&< dat) 5!:0
}}
pf =: {{
NB. Parallelise outer tines
y =. ((0 { y) 5!:0 t.'' toar) 0 } y
y =. ((2 { y) 5!:0 t.'' toar) 2 } y
NB. Open to wait for tines to finish
y =. ((1 { y) 5!:0 &> toar) 1 } y
}}
NB. adverb: convert u to atomic representation
toar =: {{
vb =. u f.
ar =. 5!:1<'vb' }}
(+/%#) psf
+/t.'' %&:> #t.''
Of course, in this simplistic example, the parallel version is slower,
because the outer tines are so simple, but on more complicated verbs, I
can
imagine it gains some speed. Note that any side-effects the outer tines
would have would become evidently unreliable as to their execution
order.
The second approach uses a tie and `:6 to create a conjunction to be
applied between middle and last tine, that returns an adverb that will
take
the left tine, and combine it into a fork. Likely it can be written
tacitly, but I did not yet spend enough time learning tacit modifiers
trains.
NB. parallel fork, conjunction.
pfr=: {{ (t.'')(`(u&:>))(`(v t.''))(`:6)}}
+/ % pfr #
+/t.'' %&:> #t.''
This luckily works fine for combinations in other verbs and even nested
application, e. g. the hypothetical (sorry, couldn't come up with an
example that made sense)
$ +/ % prf # ?
$ (+/t.'' %&> #t.'') ?
$ +/ % prf # prf ?
$t.'' (+/t.'' %&> #t.'')&> ?t.''
I guess there's something to say for both approaches. The first is
certaily
more wordy, but I think it could be extended to recursing through an AR
and
parallelising where it makes sense (based on some yet-to-be-defined
metric).
The second one requires the user to be aware of parallelisation (also
allowing more targeted application), and adapt their code to make use of
it. It also has a lot less moving parts.
I hope someone finds this useful.
Jan-Pieter
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
