It does execute them in parallel per iteration. And yes probably we should clarify this behavior in the docs.
- Bolke > Op 5 okt. 2016, om 10:00 heeft Vincent Poulain > <[email protected]> het volgende geschreven: > > But documentation says : "LocalExecutor executes tasks locally in parallel. > It uses the multiprocessing Python library and queues to parallelize the > execution of tasks." > > http://airflow.incubator.apache.org/code.html?highlight=localexecutor#airflow.executors.LocalExecutor > > On Wed, Oct 5, 2016 at 9:36 AM, Bolke de Bruin <[email protected]> wrote: > >> This is expected behavior. The LocalExecutor runs in process with the >> scheduler and it will wait on tasks that are running before iterating. >> >> If you you don’t want this switch to the CeleryExecutor. >> >> Bolke >> >> >>> Op 5 okt. 2016, om 09:18 heeft Vincent Poulain < >> [email protected]> het volgende geschreven: >>> >>> Hey guys, >>> >>> I wrote a question on StackOverflow about scheduler that does not trigger >>> DAG anymore when a DAG is running and until the DAG end up. Anyone can >> help >>> ? >>> >>> http://stackoverflow.com/questions/39862590/airflow- >> does-not-trigger-concurrent-dags-with-localexecutor >>> >>> >>> Thanks ! >>> >>> -- >>> *Vincent Poulain* >>> office: +33 9 52 57 62 09 / mobile: +33 6 21 82 87 62 >>> [email protected] >>> >>> *tinyclues */ 1 rue du Mail 75002 Paris >>> www.tinyclues.com / @tinyclues <http://www.twitter.com/tinyclues> >> >> > > > -- > *Vincent Poulain* > office: +33 9 52 57 62 09 / mobile: +33 6 21 82 87 62 > [email protected] > > *tinyclues */ 1 rue du Mail 75002 Paris > www.tinyclues.com / @tinyclues <http://www.twitter.com/tinyclues>
