[julia-users] Waiting on multiple subprocesses to finish?
I was reading the docs http://julia.readthedocs.org/en/latest/manual/control-flow/#tasks-and-events, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait() command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ;) -E
Re: [julia-users] Waiting on multiple subprocesses to finish?
Check out the implementation of multi.jl:pmap (the part in the @sync and @async blocks), it's a great example. --Tim On Thursday, July 17, 2014 04:23:48 PM Elliot Saba wrote: I was reading the docs http://julia.readthedocs.org/en/latest/manual/control-flow/#tasks-and-event s, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait() command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ;) -E
Re: [julia-users] Waiting on multiple subprocesses to finish?
Note that the tasks all run in the same thread, so if any of the tasks (or the main process) have a long running computation which does not yield, the other processes won't run. I believe that `sleep 10` falls in this category, although I'm not 100% certain. Cheers, Kevin On Thu, Jul 17, 2014 at 1:49 PM, Tim Holy tim.h...@gmail.com wrote: Check out the implementation of multi.jl:pmap (the part in the @sync and @async blocks), it's a great example. --Tim On Thursday, July 17, 2014 04:23:48 PM Elliot Saba wrote: I was reading the docs http://julia.readthedocs.org/en/latest/manual/control-flow/#tasks-and-event s, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait() command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ;) -E
Re: [julia-users] Waiting on multiple subprocesses to finish?
Thanks, I'll take a look, although I have the gut feeling that these functions are targeted toward multiple processes. -E On Thu, Jul 17, 2014 at 4:49 PM, Tim Holy tim.h...@gmail.com wrote: Check out the implementation of multi.jl:pmap (the part in the @sync and @async blocks), it's a great example. --Tim On Thursday, July 17, 2014 04:23:48 PM Elliot Saba wrote: I was reading the docs http://julia.readthedocs.org/en/latest/manual/control-flow/#tasks-and-event s, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait() command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ;) -E
Re: [julia-users] Waiting on multiple subprocesses to finish?
I'm trying to run shell commands in parallel. Shouldn't waiting for a process to finish just call wait() which then invokes the scheduler? Or did I misread the above linked paragraph completely wrong? Specifically, I'm talking about the sections that say: The basic function for waiting for an event is wait. Several objects implement wait; for example, given a Process object, wait will wait for it to exit. In the next paragraph, it says: When a task calls wait on a Condition, the task is marked as non-runnable, added to the condition’s queue, and switches to the scheduler. The scheduler will then pick another task to run, or block waiting for external events. I guess I thought that block waiting for external events would only happen when there were no other tasks to run, but perhaps what's happening is that switching to other tasks only happens when I'm waiting on a condition constructed within Julia? -E On Thu, Jul 17, 2014 at 5:25 PM, Kevin Squire kevin.squ...@gmail.com wrote: Note that the tasks all run in the same thread, so if any of the tasks (or the main process) have a long running computation which does not yield, the other processes won't run. I believe that `sleep 10` falls in this category, although I'm not 100% certain. Cheers, Kevin On Thu, Jul 17, 2014 at 1:49 PM, Tim Holy tim.h...@gmail.com wrote: Check out the implementation of multi.jl:pmap (the part in the @sync and @async blocks), it's a great example. --Tim On Thursday, July 17, 2014 04:23:48 PM Elliot Saba wrote: I was reading the docs http://julia.readthedocs.org/en/latest/manual/control-flow/#tasks-and-event s, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait() command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ;) -E
Re: [julia-users] Waiting on multiple subprocesses to finish?
you can also call `spawn` (or `open`) rather than `run` to run a process asynchronously in the current task (it returns a process handle rather than a return value): function run(cmds::AbstractCmd, args...) ps = spawn(cmds, spawn_opts_inherit(args...)...) success(ps) ? nothing : pipeline_error(ps) end On Thu, Jul 17, 2014 at 8:33 PM, Elliot Saba staticfl...@gmail.com wrote: Thanks, I'll take a look, although I have the gut feeling that these functions are targeted toward multiple processes. -E On Thu, Jul 17, 2014 at 4:49 PM, Tim Holy tim.h...@gmail.com wrote: Check out the implementation of multi.jl:pmap (the part in the @sync and @async blocks), it's a great example. --Tim On Thursday, July 17, 2014 04:23:48 PM Elliot Saba wrote: I was reading the docs http://julia.readthedocs.org/en/latest/manual/control-flow/#tasks-and-event s, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait() command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ;) -E
Re: [julia-users] Waiting on multiple subprocesses to finish?
Great! That answers it, Jameson. Thanks! On Thu, Jul 17, 2014 at 8:48 PM, Jameson Nash vtjn...@gmail.com wrote: you can also call `spawn` (or `open`) rather than `run` to run a process asynchronously in the current task (it returns a process handle rather than a return value): function run(cmds::AbstractCmd, args...) ps = spawn(cmds, spawn_opts_inherit(args...)...) success(ps) ? nothing : pipeline_error(ps) end On Thu, Jul 17, 2014 at 8:33 PM, Elliot Saba staticfl...@gmail.com wrote: Thanks, I'll take a look, although I have the gut feeling that these functions are targeted toward multiple processes. -E On Thu, Jul 17, 2014 at 4:49 PM, Tim Holy tim.h...@gmail.com wrote: Check out the implementation of multi.jl:pmap (the part in the @sync and @async blocks), it's a great example. --Tim On Thursday, July 17, 2014 04:23:48 PM Elliot Saba wrote: I was reading the docs http://julia.readthedocs.org/en/latest/manual/control-flow/#tasks-and-event s, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait() command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ;) -E
