tirkarthi commented on PR #46820:
URL: https://github.com/apache/airflow/pull/46820#issuecomment-2665894739
Thanks @jscheffl for the review. Glad to have the points since you
implemented it in Airflow 2.
> The syntax with :: is the Github style. The variant with ## is the way how
ADO (MS AzureDevOps Pipelines) handle log grouping. As I was contributing this
initially I wanted to make it compatible to both. If this is major complexity
we can re-discuss. Else I'd propose to keep it.
Supporting Azure would involve using regex and another branch along with
handling of mix match between github and azure which will make this
implementation complex.
> Before I manually test, does it support nested grouping? (Github does not
and I was always proud Airflow does. Code looks like but have not checked.
Nested groups are unfortunately not supported at the moment since I go
through each line and then add it to an array to empty it once the group ends.
Nested groups might take something like a stack of lines to push and pop along
with embedding the summary and details tag correctly inside. I have added a
todo about the same.
I used the below dag to generate logs of various sizes from 1 to 80 MB and I
don't see unexpected delay other than few seconds of delay to render the larger
log files in the UI along with downloading. Speaking of this there used to be a
limit for larger logs in old UI with a download button to view it. Download
button was also handy tool for offline views and analysis which is missing in
the new UI currently.
I guess this is good enough for an initial implementation and can always be
iterated through later since we use this feature very much internally to handle
logs.
```python
from datetime import datetime
import uuid
from airflow import DAG
from airflow.decorators import task
with DAG(
dag_id="log_grouping",
start_date=datetime(2025, 2, 1),
catchup=False,
schedule=None,
) as dag:
@task
def generate(**context):
line_limit = int(context["params"]["line_limit"])
group_limit = int(context["params"]["group_limit"])
print("::group::group name")
for index in range(group_limit):
print(f"::group::inner group name {index}")
for index in range(line_limit):
print(" ".join([uuid.uuid4().hex] * 1))
print("::endgroup::")
print("::endgroup::")
generate()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
