errose28 commented on PR #3648: URL: https://github.com/apache/ozone/pull/3648#issuecomment-1209650526
Thanks for the input @ChenSammi. The subtasks in [HDDS-6449](https://issues.apache.org/jira/browse/HDDS-6449) were actually created before we had design discussions. There won't need to be a background service which greatly simplifies the task and it could probably be done in one Jira. But I agree that the progress on that task is uncertain and we have a proposal to fix the import issue now, so let's go ahead and introduce the new directory here. Some thoughts on introducing the new directory: - I think we should have one tmp working directory per volume, with different subdirectories for different tasks as you mentioned. Container delete, import, and create would be examples, although we only need to handle import in this jira. - Containers should be moved from the tmp directory to their intended location using a directory rename atomic at the FS level. - IMO `hdds.datanode.replication.work.dir` should be deprecated and ignored. Datanodes require a clean disk state in their container directories on restart (see HDDS-6441 and HDDS-6449), so any non-atomic FS operation that could compromise that like RocksDB creation (schema < V3) copy from import directory, or delete must be done from an atomic directory rename on the same filesystem. - The tmp directory needs to be chosen with care. - Currently all volume state is contained in the `hdds` subdirectory of each volume directory/disk mount. I think this feature should maintain that practice. - We cannot add a new subdirectory immediately under `hdds` as this will break downgrade without an upgrade layout feature. Ozone 1.1.0 has a check that `hdds` directory only has one subdirectory. - IMO `hdds/<clusterID>/tmp` would be the best place to put it, but we need to make sure this will not affect datanode startup on downgrade. If it does and there is no better directory, we may need to add a simple HDDS layout feature for this change. Let me know your thoughts on choosing a location for the tmp directory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
