o-nikolas commented on issue #43717: URL: https://github.com/apache/airflow/issues/43717#issuecomment-2711354128
> > Thanks for the info [@mjordyn](https://github.com/mjordyn)! Can you share a snippet of where you're putting the sleep in your ECS task? It sounds like you're putting it at the beginning of your task, then your task does it's actual work, then you see the logs? Or did you replace the actual work with only a sleep? If you have an actual code snipped (or if you could reproduce what change you made) and share it, that would be tremendous :) > > Here's a highly simplified/psuedocode version of the code for one of the tasks with the sleep addition: > > ``` > import logging > import argparse > import time > import anotherpackage > import asecondpackage > > logging.basicConfig(level="INFO") > logger = logging.getLogger(__name__) > > def main(): > parser = argparse.ArgumentParser() > parser.add_argument('-a', '--arg_a', help='Arg a') > parser.add_argument('-b', '--arg_b', help='Arg b') > parser.add_argument('-c', '--arg_c', help='Arg c') > parser.add_argument('-d', '--arg_d', help='Arg d') > args = vars(parser.parse_args()) > > time.sleep(10) > > logger.info("Starting task") > logger.info("Logging something else") > > var1 = args['key1'] > var2 = args['key2'] > var3 = args['key3'] > start_time = time.time() > > client = get_client() > > files = get_file_list() > file_count = len(files) > for i, file in enumerate(files): > if i % 100 == 0: > logger.info(f"{(i/file_count) * 100} percent complete after {(time.time()-start_time)/60} minutes") > process_file(file, client) > > logger.info("Finished task") > ``` > > The only difference between the first few logs showing up in the UI or not is whether or not the `time.sleep(10)` line was present or not. That sleep did not replace any actual processing code, it was purely an addition. When the sleep line was present, everything showed up in the Airflow UI. When it wasn't, the first 2 `logger.info` messages did not appear. The first `logger.info` message within the loop sometimes did not show up (and sometimes did), but all messages after that always did. Regardless of whether or not the sleep statement was present, all logs always appeared in the ECS task Cloudwatch logs. Thanks for the details! Got it, so you updated the python code that runs inside the ECS container that you trigger with the EcsRunTaskOperator. Your issue seems to be different from @yaningz, they are missing the tail of their logs, and you are missing the head. I'm still unable to reproduce either case. What infrastructure are you running Airflow on? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
