Hi,
Along with celery worker, you might have to configure dag concurrency
and number of celery workers to get maximum parallelism in airflow. As long
as scheduler has enough resources to schedule tasks and enough workers to
run the job, you can parallely run a lot of tasks in airflow.
On Tue 11 Dec, 2018, 08:32 lkes...@gmail.com <lkes...@gmail.com wrote:
> Hi,
>
> Can you please guide us how to run the tasks parallely in airflow as
> which i need to run almost 14 tasks parallely. In config file executor as
> 'celery'. Is there any thing which i need to concentrate or guide me to run
> the tasks.
>
>
> thx,
> Lokesh
>
