Lee-W commented on code in PR #54054:
URL: https://github.com/apache/airflow/pull/54054#discussion_r2567372268
##########
providers/amazon/src/airflow/providers/amazon/aws/log/cloudwatch_task_handler.py:
##########
@@ -163,20 +171,35 @@ def upload(self, path: os.PathLike | str, ti: RuntimeTI):
self.close()
return
- def read(self, relative_path, ti: RuntimeTI) -> tuple[LogSourceInfo,
LogMessages | None]:
- logs: LogMessages | None = []
+ def read(self, relative_path: str, ti: RuntimeTI) -> LogResponse:
+ messages, logs = self.stream(relative_path, ti)
+ str_logs: list[str] = []
+
+ for group in logs:
+ for msg in group:
+ str_logs.append(f"{msg}\n")
Review Comment:
```suggestion
str_logs: list[str] = [
f"{msg}]n"
for group in logs
for msg in group
]
```
##########
providers/amazon/src/airflow/providers/amazon/aws/log/cloudwatch_task_handler.py:
##########
@@ -192,29 +215,22 @@ def get_cloudwatch_logs(self, stream_name: str,
task_instance: RuntimeTI):
if (end_date := getattr(task_instance, "end_date", None)) is None
else datetime_to_epoch_utc_ms(end_date + timedelta(seconds=30))
)
- events = self.hook.get_log_events(
+ return self.hook.get_log_events(
log_group=self.log_group,
log_stream_name=stream_name,
end_time=end_time,
)
- return "\n".join(self._event_to_str(event) for event in events)
- def _event_to_dict(self, event: dict) -> dict:
+ def _parse_cloudwatch_log_event(self, event: CloudWatchLogEvent) -> str:
Review Comment:
```suggestion
def _parse_log_event_as_dumped_json(self, event: CloudWatchLogEvent) ->
str:
```
would this be better this way? i would expect it to be a dict at first glance
##########
providers/amazon/src/airflow/providers/amazon/aws/log/cloudwatch_task_handler.py:
##########
@@ -192,29 +215,22 @@ def get_cloudwatch_logs(self, stream_name: str,
task_instance: RuntimeTI):
if (end_date := getattr(task_instance, "end_date", None)) is None
else datetime_to_epoch_utc_ms(end_date + timedelta(seconds=30))
)
- events = self.hook.get_log_events(
+ return self.hook.get_log_events(
log_group=self.log_group,
log_stream_name=stream_name,
end_time=end_time,
)
- return "\n".join(self._event_to_str(event) for event in events)
- def _event_to_dict(self, event: dict) -> dict:
+ def _parse_cloudwatch_log_event(self, event: CloudWatchLogEvent) -> str:
Review Comment:
or there's a `_event_to_str` in `CloudwatchTaskHandler`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
