potiuk commented on issue #27345: URL: https://github.com/apache/airflow/issues/27345#issuecomment-1295101438
Are you sure ALL messages are duplicated? There was a change to reveal messages produced during parsing which were hidden, and this is something that coudl cause more logs - but that does not mean every log is repeated - https://github.com/apache/airflow/pull/26779 If my quess is right - you will get duplicated logs from parsing but not from execution of the tasks and if you help to verify that this is the case - the silution is for you to sort out the messages according to where they are coming from (because they might be relevant and possibly different on execution and different on parsing (even if some of them are repeated). Also it might mean that you are doing "too much" during parsing at the top-level code of your DAG. Can you post examples of such duplicated lines and see if the actual execuition is NOT duplicated ?| -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
