RNHTTR commented on code in PR #38807:
URL: https://github.com/apache/airflow/pull/38807#discussion_r1556051546
##########
airflow/providers/amazon/aws/log/s3_task_handler.py:
##########
@@ -178,7 +180,13 @@ def s3_read(self, remote_log_location: str, return_error:
bool = False) -> str:
:return: the log found at the remote_log_location
"""
try:
- return self.hook.read_key(remote_log_location)
+ range: str = None
+ if page_number is not None:
+ page_size = 1024 * 100 # TODO: Create config for page_size
Review Comment:
> Does it even need to be API paremeters, or could we do what S3 does, and
do this as an HTTP Range request?
This is currently the plan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
