Ethanlm commented on a change in pull request #3094: STORM-3477 switch HDFS
blobstore isRemoteBlobExists to use file exist…
URL: https://github.com/apache/storm/pull/3094#discussion_r308417703
##########
File path:
external/storm-hdfs-blobstore/src/main/java/org/apache/storm/hdfs/blobstore/HdfsBlobStore.java
##########
@@ -336,6 +336,27 @@ public InputStreamWithMeta getBlob(String key, Subject
who)
}
}
+ /**
+ * Checks if a blob exists.
+ *
+ * @param key blobstore key
+ * @param who subject
+ * @throws AuthorizationException if authorization is failed
+ * @throws KeyNotFoundException if key is not found
+ */
+ public boolean blobExists(String key, Subject who) throws
AuthorizationException, KeyNotFoundException {
+ who = checkAndGetSubject(who);
+ validateKey(key);
+ SettableBlobMeta meta = getStoredBlobMeta(key);
+ aclHandler.hasPermissions(meta.get_acl(), READ, who, key);
+ try {
+ BlobStoreFile blobStoreFile = hbs.read(DATA_PREFIX + key);
+ return blobStoreFile.exists();
Review comment:
We don't need this.
For every key, there will be two files `meta_<key>` and `data_<key>`. Every
operation to the blob will first check and get `meta_<key>`. If the blob is
deleted, it throws `KeyNotFoundException`. If it exists, we don't need to
continue to check `data_<key>` because we can simply assume `data_<key>`
exists. Because if `meta_<key>`exists but `data_<key>` doesn't, there will be
RuntimeException with the client tries to download or access to the blob.
In this way, we can reduce the hdfs operation further.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
