mynameborat commented on a change in pull request #1236: SAMZA-2419: removing
duplicate cleaning of stores
URL: https://github.com/apache/samza/pull/1236#discussion_r365013346
##########
File path:
samza-core/src/main/java/org/apache/samza/storage/TransactionalStateTaskRestoreManager.java
##########
@@ -243,30 +243,19 @@ static StoreActions getStoreActions(
timeSinceLastCheckpointInMs = System.currentTimeMillis() -
checkpointedChangelogOffset.getCheckpointId().getMillis();
}
-
- // if the clean.store.start config is set, delete the currentDir,
restore from oldest offset to checkpointed
- if (storageEngine.getStoreProperties().isPersistedToDisk() && new
StorageConfig(
- config).getCleanLoggedStoreDirsOnStart(storeName)) {
- File currentDir =
storageManagerUtil.getTaskStoreDir(nonLoggedStoreBaseDirectory, storeName,
taskName, taskMode);
- LOG.info("Marking current directory: {} for store: {} in task: {}.",
currentDir, storeName, taskName);
- storeDirsToDelete.put(storeName, currentDir);
- LOG.info("Marking restore offsets for store: {} in task: {} to {},
{} ", storeName, taskName, oldestOffset, checkpointedOffset);
- storesToRestore.put(storeName, new RestoreOffsets(oldestOffset,
checkpointedOffset));
- return;
- }
// if the clean.store.start config is set, delete the currentDir,
restore from oldest offset to checkpointed
if (storageEngine.getStoreProperties().isPersistedToDisk() && new
StorageConfig(
Review comment:
I'd suggest you also check if the store is logged store explicitly. It is
possible logged store directory and non-logged store directory happens to be
the same directory and we will by-mistake clear the stores that are non-logged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
