gianm commented on code in PR #13955:
URL: https://github.com/apache/druid/pull/13955#discussion_r1149844989
##########
extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java:
##########
@@ -66,6 +67,11 @@ public InputStream get(final String bucket, final String
path, long start) throw
return inputStream;
}
+ public StorageObject getMetadata(final String bucket, final String path)
throws IOException
+ {
+ return storage.get().objects().get(bucket, path).execute();
Review Comment:
> Other paths retry. Should this one?
Hmm. It seems like some other paths retry and some don't. The Google SDK has
built-in and configurable retries:
https://cloud.google.com/storage/docs/retry-strategy#java. We don't customize
this when creating SDK clients, and the defaults do provide for what looks like
a reasonable level of retries (6 attempts with backoff over about a minute). So
that suggests we don't need to do our own.
> What happens if the object does not exist? Is the intent to fail the
ingest? Or, skip missing files? If skip, how will the user know which files
were skipped?
If the object doesn't exist, the intent is that the ingest fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
