carloea2 commented on code in PR #4136:
URL: https://github.com/apache/texera/pull/4136#discussion_r2641408732
##########
common/workflow-core/src/main/scala/org/apache/texera/service/util/S3StorageClient.scala:
##########
@@ -259,4 +262,79 @@ object S3StorageClient {
DeleteObjectRequest.builder().bucket(bucketName).key(objectKey).build()
)
}
+ def uploadPart(
+ bucket: String,
+ key: String,
+ uploadId: String,
+ partNumber: Int,
+ inputStream: InputStream,
+ contentLength: Option[Long]
+ ): Unit = {
+ val body: RequestBody = contentLength match {
Review Comment:
Yes, for the case when user does not specify Content Length, we read all the
bytes. (However this case is forbidden in uploadPart endpoint)
The case when user specify Content Length;
`RequestBody.fromInputStream(inputStream, contentLength /* = ex 5 GiB */) `the
SDK does not read and buffer the whole 5 GiB in memory first. For retries
(depends in support), the SDK tries rewinding by using InputStream.reset() with
a read limit of 128 KiB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
