tirkarthi commented on PR #39006:
URL: https://github.com/apache/airflow/pull/39006#issuecomment-2066933356
@dirrao @jscheffl Used below dag which generated around 30MB log file and
the function call added around 15ms with total render time taking 9-10 seconds.
I did it with dev builds using "yarn dev" since the function name was not
searchable in production mode possibly due to no source mapping. Please let me
know if I missed anything. Thanks
```pythonfrom __future__ import annotations
from datetime import datetime
from airflow import DAG
from airflow.decorators import task
with DAG(
dag_id="perf_39006",
start_date=datetime(2024, 1, 1),
catchup=False,
schedule_interval=None,
) as dag:
@task
def log_line_generator():
import random
lorem_ipsum_words = "Lorem ipsum dolor sit amet, consectetur
adipiscing elit, sed do eiusmod tempor".split()
for _ in range(200_000):
random.shuffle(lorem_ipsum_words)
line = " ".join(lorem_ipsum_words)
rand = random.random()
if rand > 0.9:
print(f"{line} error")
elif rand > 0.8:
print(f"{line} warn")
else:
print(line)
log_line_generator()
```
Without patch
With patch :
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
