[
https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769086#comment-17769086
]
Arpit Goyal commented on KAFKA-15169:
-------------------------------------
Thanks [~divijvaidya]
I have two questions based on the code walkthrough
{code:java}
private <T> T loadIndexFile(File file, RemoteLogSegmentMetadata
remoteLogSegmentMetadata,
Function<RemoteLogSegmentMetadata, InputStream>
fetchRemoteIndex,
Function<File, T> readIndex) throws IOException
{
File indexFile = new File(cacheDir, file.getName());
T index = null;
if (Files.exists(indexFile.toPath())) {
try {
index = readIndex.apply(indexFile);
} catch (CorruptRecordException ex) {
log.info("Error occurred while loading the stored index file
{}", indexFile.getPath(), ex);
}
}
if (index == null) {
File tmpIndexFile = new File(indexFile.getParentFile(),
indexFile.getName() + RemoteIndexCache.TMP_FILE_SUFFIX);
try (InputStream inputStream =
fetchRemoteIndex.apply(remoteLogSegmentMetadata)) {
Files.copy(inputStream, tmpIndexFile.toPath(),
StandardCopyOption.REPLACE_EXISTING);
}
Utils.atomicMoveWithFallback(tmpIndexFile.toPath(),
indexFile.toPath(), false);
index = readIndex.apply(indexFile);
}
return index;
}
{code}
In the RemoteIndexCache (loadIndexFile) function
1. First we check if file exists on the disk and do a sanityCheck. I believe
this part of code will never be executed as it occurs only when there is a
cache miss operation.
2. As per the first test case it would through Corrupt record exception at the
later part of the code where we fetch it from remote segment and doing a
sanityCheck
{code:java}
Utils.atomicMoveWithFallback(tmpIndexFile.toPath(), indexFile.toPath(), false);
index = readIndex.apply(indexFile);
{code}
I was believing the first test case was related to file already exist on the
disk and then call getIndexEntry
1. Create a empty/corrupt file on disk
2. Call getIndexEntry
3. It throws record corrupted action
4. In the next line it fetches from remote storage and restore the file.
> Add tests for RemoteIndexCache
> ------------------------------
>
> Key: KAFKA-15169
> URL: https://issues.apache.org/jira/browse/KAFKA-15169
> Project: Kafka
> Issue Type: Test
> Reporter: Satish Duggana
> Assignee: Arpit Goyal
> Priority: Major
> Labels: KIP-405
> Fix For: 3.7.0
>
>
> Follow-up from
> https://github.com/apache/kafka/pull/13275#discussion_r1257490978
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
