anoopsjohn commented on a change in pull request #633: HBASE-22890 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/633#discussion_r325765169
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java
##########
@@ -59,6 +65,30 @@ public FileIOEngine(long capacity, String... filePaths)
throws IOException {
this.fileChannels = new FileChannel[filePaths.length];
this.rafs = new RandomAccessFile[filePaths.length];
this.channelLocks = new ReentrantLock[filePaths.length];
+ init();
+ }
+
+ @Override
+ public boolean verifyFileIntegrity(byte[] persistentChecksum, String
algorithm)
+ throws IOException {
+ try {
+ byte[] calculateChecksum = calculateChecksum(algorithm);
+ if (!Bytes.equals(persistentChecksum, calculateChecksum)) {
+ throw new IOException("The persistent checksum is " +
Bytes.toString(persistentChecksum) +
+ ", but the calculate checksum is " +
Bytes.toString(calculateChecksum));
+ }
+ return true;
+ } catch (IOException ioex) {
+ LOG.error("File verification failed because of ", ioex);
+ reinit();
+ return false;
+ } catch (NoSuchAlgorithmException nsae) {
+ LOG.error("No such algorithm " + algorithm, nsae);
+ throw new RuntimeException(nsae);
Review comment:
What will happen if we throw RTE here? The FileIOE create itself will get
failed or even more? Retrieve from persistent cached fail need not be a reason
for the FileIOE to be failed. But here we have configured this to be a
persistent FileIOE. Later when we try persist during RS shutdown, we will get
failure any way (?). Are u considering that here for this RTE? Am not really
sure this throw RTE is correct here. Thoughts
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
