bkonold commented on a change in pull request #1236: SAMZA-2419: removing
duplicate cleaning of stores
URL: https://github.com/apache/samza/pull/1236#discussion_r378590970
##########
File path:
samza-core/src/main/java/org/apache/samza/storage/TransactionalStateTaskRestoreManager.java
##########
@@ -243,30 +243,19 @@ static StoreActions getStoreActions(
timeSinceLastCheckpointInMs = System.currentTimeMillis() -
checkpointedChangelogOffset.getCheckpointId().getMillis();
}
-
- // if the clean.store.start config is set, delete the currentDir,
restore from oldest offset to checkpointed
- if (storageEngine.getStoreProperties().isPersistedToDisk() && new
StorageConfig(
- config).getCleanLoggedStoreDirsOnStart(storeName)) {
- File currentDir =
storageManagerUtil.getTaskStoreDir(nonLoggedStoreBaseDirectory, storeName,
taskName, taskMode);
- LOG.info("Marking current directory: {} for store: {} in task: {}.",
currentDir, storeName, taskName);
- storeDirsToDelete.put(storeName, currentDir);
- LOG.info("Marking restore offsets for store: {} in task: {} to {},
{} ", storeName, taskName, oldestOffset, checkpointedOffset);
- storesToRestore.put(storeName, new RestoreOffsets(oldestOffset,
checkpointedOffset));
- return;
- }
// if the clean.store.start config is set, delete the currentDir,
restore from oldest offset to checkpointed
if (storageEngine.getStoreProperties().isPersistedToDisk() && new
StorageConfig(
Review comment:
to recap: @mynameborat 's question was with regard to side inputs, which are
persisted and non-logged but are not to be deleted. this is a non-issue since
side inputs are not hooked in through TransactionalStateTaskRestoreManager but
instead TaskSideInputStorageManager. @mynameborat feel to comment if i am
misunderstanding our discussion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
