ephraimbuddy commented on code in PR #26750:
URL: https://github.com/apache/airflow/pull/26750#discussion_r993821390
##########
airflow/cli/commands/task_command.py:
##########
@@ -364,14 +363,7 @@ def task_run(args, dag=None):
print(f'Loading pickle id: {args.pickle}')
dag = get_dag_by_pickle(args.pickle)
elif not dag:
- if args.local:
Review Comment:
> hi @ephraimbuddy could you please give more context about why this should
be removed?
>
> it looks like your PR only tried to optimize when executing a task via
fork. there is also an important part
https://github.com/apache/airflow/pull/26750/files#diff-8b336b0706abbada03e8dce519e122384da2c913cc79ee4d638d0427a1342d41L47-L48
>
> i think we should need `dag = get_dag_by_deserialization(args.dag_id)`
please let me know your thoughts
We noticed a significant increase in task run duration with the previous
change. Using `get_dag_by_deserialization` only delays the dag parsing(for
`_start_by_fork`). Looking at `exec` I think we should devise another way to
get the serialized dag for use because using the previous change, we had to
parse the dag in TaskRunner process for `fork` which causes delays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
