gianm commented on code in PR #16236:
URL: https://github.com/apache/druid/pull/16236#discussion_r1551890865
##########
extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java:
##########
@@ -53,20 +62,34 @@ public class GoogleStorage
* <p>
* See OmniDataSegmentKiller for how DataSegmentKillers are initialized.
*/
- private static final Logger log = new Logger(GoogleStorage.class);
-
private final Supplier<Storage> storage;
- private final HumanReadableBytes DEFAULT_WRITE_CHUNK_SIZE = new
HumanReadableBytes("4MiB");
-
public GoogleStorage(final Supplier<Storage> storage)
{
this.storage = storage;
}
- public void insert(final String bucket, final String path,
AbstractInputStreamContent mediaContent) throws IOException
+ /**
+ * Upload an object. From {@link Storage#createFrom(BlobInfo, InputStream,
int, Storage.BlobWriteOption...)},
+ * "larger buffer sizes might improve the upload performance but require
more memory."
+ *
+ * @param bucket target bucket
+ * @param path target path
+ * @param mediaContent content to upload
+ * @param bufferSize size of upload buffer, or {@link #DEFAULT_BUFFER_SIZE}
+ */
+ public void insert(
+ final String bucket,
+ final String path,
+ final AbstractInputStreamContent mediaContent,
+ final int bufferSize
+ ) throws IOException
{
- storage.get().createFrom(getBlobInfo(bucket, path),
mediaContent.getInputStream());
+ if (bufferSize == DEFAULT_BUFFER_SIZE) {
Review Comment:
"Buffer size" is the term the Google API client uses for this parameter. It
looks like it uses "chunk size" for the other parameter you mention. IMO, we
should retain the terms the Google client uses, even though they aren't
consistent with each other. Better to be consistent with the upstream terms.
As to the way the default gets picked, I'll change the `bufferSize` to be a
`@Nullable Integer` to match the way `chunkSize` works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
