[
https://issues.apache.org/jira/browse/BEAM-13827?focusedWorklogId=721843&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-721843
]
ASF GitHub Bot logged work on BEAM-13827:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Feb/22 11:03
Start Date: 07/Feb/22 11:03
Worklog Time Spent: 10m
Work Description: hoshimura commented on pull request #16735:
URL: https://github.com/apache/beam/pull/16735#issuecomment-1031340508
@charlesccychen So, the current implementation will raise an error for files
of size > MIN_WRITE_SIZE = 5 * 1024 * 1024 but < MAX_WRITE_SIZE = 5 * 1024 *
1024 * 1024.
because the while case in put() of s3io.py will then be true for one
iteration causing the resulting self.buffer to be of zero length. This has the
implication that finish() will fail with a client error since no check was made
for zero length data.
I don't know if there has to be some record of that put has written data
already so that an error in finish() will only be interpreted as an error for
true zerolength files but not for files with a zerolength buffer when reaching
finish()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 721843)
Time Spent: 40m (was: 0.5h)
> upload to s3 fails if last chunk in finish happens to be of 0 length
> --------------------------------------------------------------------
>
> Key: BEAM-13827
> URL: https://issues.apache.org/jira/browse/BEAM-13827
> Project: Beam
> Issue Type: Bug
> Components: io-py-aws
> Affects Versions: 2.35.0
> Reporter: Johan Sternby
> Assignee: Johan Sternby
> Priority: P2
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Multipart upload to s3 fails in finish method when last chunk is set to zero
> length, this happens if original buffer size is betweenÂ
> MIN_WRITE_SIZE = 5 * 1024 * 1024
> MAX_WRITE_SIZE = 5 * 1024 * 1024 * 1024
> from put()
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
