rawwar commented on issue #44694:
URL: https://github.com/apache/airflow/issues/44694#issuecomment-2673565088
@ferruzzi , after a lot of random changes, here's what's going on.
```
for response in paginator.paginate(
logGroupName=log_group,
logStreamNames=[run_id],
PaginationConfig={"StartingToken": continuation_token},
):
```
Here', the paginator is continuously returning empty events as below:
```
paginator response {'events': [], 'searchedLogStreams': [], 'nextToken':
'<NEXT TOKEN>', 'ResponseMetadata': {'RequestId':
'3929ba0b-fe0d-4528-927b-2f61348235fa', 'HTTPStatusCode': 200, 'HTTPHeaders':
{'x-amzn-requestid': '3929ba0b-fe0d-4528-927b-2f61348235fa', 'content-type':
'application/x-amz-json-1.1', 'content-length': '414', 'date': 'Tue, 03 Dec
2024 12:07:14 GMT'}, 'RetryAttempts': 0}}
```
They keep going on. I've seen two cases. In one case, it actually gets the
glue logs and prints them after an hour or so. In another case, I see following
errors:
```
botocore.exceptions.ClientError: An error occurred
(InvalidSignatureException) when calling the FilterLogEvents operation:
Signature expired: 20241203T121753Z is now earlier than 20241203T124147Z
(20241203T124647Z - 5 min.)
```
Adding startTime to paginator.paginate fixed the issue. Here's the updated
code that worked. I've set startTime to 24 hours prior
```
for response in paginator.paginate(
logGroupName=log_group,
logStreamNames=[run_id],
startTime=(datetime.now() - timedelta(hours=24)), # 24
hours ago
PaginationConfig={"StartingToken": continuation_token},
):
```
Setting startTime is successfully fetching logs. Can we hardcode startTime
of the task?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
