I'm new to Julia, so I'm not 100% confident on this solution, but here's
what I came up with:
@everywhere function printfile(file)
println(myid(), " ", file)
end
np = nprocs()
@sync begin
producer = @task file_producer(library_root)
for p = 1:np
if p != myid() || np == 1
@async begin
for file in producer
result = remotecall_fetch(p, printfile, file)
end
end
end
end
end
On Wednesday, October 15, 2014 10:36:17 AM UTC-6, Duane Johnson wrote:
>
> I'm attempting the following producer/consumer pattern, but I'm trying to
> give the work to separate processes to maximize CPU resources:
>
> function file_producer(library_root)
> files(library_root) do f
> produce(f)
> end
> produce(END_FILE_LIST)
> end
>
>
> @parallel for f in Task(file_producer)
> println(myid(), f)
> end
>
>
> When I get to the @parallel part, it fails:
>
> ERROR: `length` has no method matching length(::Task)
> in include at /Applications/Julia-0.3.1.app/Contents/Resources/julia/lib/
> julia/sys.dylib
> in include_from_node1 at loading.jl:128
> in process_options at /Applications/Julia-0.3.1.app/Contents/Resources/
> julia/lib/julia/sys.dylib
> in _start at /Applications/Julia-0.3.1.app/Contents/Resources/julia/lib/
> julia/sys.dylib (repeats 2 times)
> while loading /Users/duane/Dropbox/Projects/wordtree/TextGrams/baseline.jl
> , in expression starting on line 1488
>
>
> Is there an easy way to let multiple processes consume work like this?
>
>
