OK, I found a horrible work around that might be good enough (for me):
Here is a mock python script:
import math, time
x = math.sin(math.pi/4)
time.sleep(5) # mock local work
print(x) # spit it out to julia
and here is the julia code that runs it:
r = @spawn readall(`python example.py`)
sleep(2) # mock local work
wait(r) # wait on python
t = fetch(r)
x = parse(t)
x - sin(pi/4) # not zero but it works...
This seems to work no matter what. It’s horrid, but better than nothing.
On Friday, October 30, 2015 at 11:14:18 PM UTC+10, Matthew Pearce wrote:
> So I got something working for my pylab example.
>
> julia> import PyCall
>
> julia> PyCall.@pyimport pylab
>
> julia> @everywhere import PyCall
>
> julia> @everywhere PyCall.@pyimport pylab
>
> julia> @everywhere A = pylab.cumsum(collect(1:10))*1.
>
> julia> fetch(@spawnat remotes[1] A)
> 10-element Array{Float64,1}:
> 1.0
> 3.0
> 6.0
> 10.0
> 15.0
> 21.0
> 28.0
> 36.0
> 45.0
> 55.0
>
>
>
>
> No luck with the math module I'm afraid. Two different types of errors
> depending on style:
>
> julia> @spawnat remotes[1] PyCall.@pyimport math as pymath
> RemoteRef{Channel{Any}}(2,1,305)
>
> julia> fetch(@spawnat remotes[1] (pymath.sin(pymath.pi / 4) - sin(pymath.pi
> / 4)) )
> ERROR: On worker 2:
> UndefVarError: pymath not defined
> in anonymous at multi.jl:1330
> in anonymous at multi.jl:889
> in run_work_thunk at multi.jl:645
> in run_work_thunk at multi.jl:654
> in anonymous at task.jl:54
> in remotecall_fetch at multi.jl:731
> [inlined code] from multi.jl:368
> in call_on_owner at multi.jl:776
> in fetch at multi.jl:784
>
> julia> @everywhere PyCall.@pyimport math as pymath
>
> julia> fetch(@spawnat remotes[1] (pymath.sin(pymath.pi / 4) - sin(pymath.pi
> / 4)) )
> Worker 2 terminated.srun: error: mrc-bsu-tesla1: task 0: Exited with exit
> code 1
> ERROR: ProcessExitedException()
> in yieldto at ./task.jl:67
> in wait at ./task.jl:367
> in wait at ./task.jl:282
> in wait at ./channels.jl:97
> in take! at ./channels.jl:84
> in take! at ./multi.jl:792
> in remotecall_fetch at multi.jl:729
> [inlined code] from multi.jl:368
> in call_on_owner at multi.jl:776
> in fetch at multi.jl:784
>
>
> ERROR (unhandled task failure): EOFError: read end of file
>
>
>
>
>
> On Friday, October 30, 2015 at 1:28:21 AM UTC, Yakir Gagnon wrote:
>>
>> @Matthew: did you find a solution?
>>
>> On Tuesday, October 27, 2015 at 8:44:53 AM UTC+10, Yakir Gagnon wrote:
>>>
>>> Yea, right? So what’s the answer? How can we if at all do any PyCalls
>>> parallely?
>>>
>>> On Monday, October 26, 2015 at 11:49:35 PM UTC+10, Matthew Pearce wrote:
>>>
>>> Thought I had an idea about this, I was wrong:
>>>>
>>>> ```julia
>>>>
>>>> julia> @everywhere using PyCall
>>>>
>>>> julia> @everywhere @pyimport pylab
>>>>
>>>> julia> remotecall_fetch(pylab.cumsum, 5, collect(1:10))
>>>> ERROR: cannot serialize a pointer
>>>> [inlined code] from error.jl:21
>>>> in serialize at serialize.jl:420
>>>> [inlined code] from dict.jl:372
>>>> in serialize at serialize.jl:428
>>>> in serialize at serialize.jl:310
>>>> in serialize at serialize.jl:420 (repeats 2 times)
>>>> in serialize at serialize.jl:302
>>>> in serialize at serialize.jl:420
>>>> [inlined code] from dict.jl:372
>>>> in serialize at serialize.jl:428
>>>> in serialize at serialize.jl:310
>>>> in serialize at serialize.jl:420 (repeats 2 times)
>>>> in serialize at serialize.jl:302
>>>> in serialize at serialize.jl:420
>>>> [inlined code] from dict.jl:372
>>>> in send_msg_ at multi.jl:222
>>>> [inlined code] from multi.jl:177
>>>> in remotecall_fetch at multi.jl:728
>>>> [inlined code] from multi.jl:368
>>>> in remotecall_fetch at multi.jl:734
>>>>
>>>> julia> pylab.cumsum(collect(1:10))
>>>> 10-element Array{Int64,1}:
>>>> 1
>>>> 3
>>>> 6
>>>> 10
>>>> 15
>>>> 21
>>>> 28
>>>> 36
>>>> 45
>>>> 55
>>>>
>>>> ```
>>>>
>>>
>>>
>>
