dazza-codes commented on issue #6750: [WIP] [AIRFLOW-6329] Add AirflowExecutor 
and native worker
URL: https://github.com/apache/airflow/pull/6750#issuecomment-570740844
 
 
   > What I would really love is to change the "type" of task message. Instead 
of sending `["airflow", "tasks", "run", ...]` it would be nice to send 
`{'dag_id': dag_id, 'task_id': task_id, ... }`
   
   As an aside, while considering options for an asyncio executor, there could 
be various ways that the task "type" (data/spec/payload - you name it) needs to 
be parsed by some kind of asyncio coroutine that can spawn and manage 1000's of 
coroutines on an event loop, such that each of those task coroutines can 
`await` any awaitable to yield control back to the event loop.  I'd like to 
understand any changes to task designs.  I've kinda bumped into this while 
trying to do a small POC on the dask executor, i.e. #6984 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to