[
https://issues.apache.org/jira/browse/RATIS-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18093034#comment-18093034
]
Tsz-wo Sze edited comment on RATIS-2581 at 7/1/26 7:16 PM:
-----------------------------------------------------------
{quote}... some downstream projects embedding Ratis need to integrate with
their own compliant file destruction mechanism.
{quote}
I was reviewing the PR and learned that Java has a
[FileSystemProvider|https://docs.oracle.com/javase/8/docs/api/java/nio/file/spi/FileSystemProvider.html]
SPI. We may
- Implement a SecureFileSystemProvider which extends FileSystemProvider.
-- Overrides the delete(..) and deleteIfExists(..) methods with the new secure
delete mechanism.
-- Overrides the newFileChannel(..) method to return new SecureFileChannel
implementation.
--- SecureFileChannel: overrides the truncate(..) method and delegate the
other calls to the original FileChannel implementation.
-- Overrides the remaining methods to delegate the calls to the original
FileSystemProvider.
Everything can be done without changing Ratis. Would this work for you?
(I was open to add a new File API and had filed RATIS-2585 for this JIRA.
But then learned that this JIRA might not be needed.)
was (Author: szetszwo):
{quote}... some downstream projects embedding Ratis need to integrate with
their own compliant file destruction mechanism.
{quote}
I was reviewing the PR and learned that Java has a FileSystemProvider SPI. We
may
- Implement a SecureFileSystemProvider which extends FileSystemProvider.
-- Overrides the delete(..) and deleteIfExists(..) methods with the new secure
delete mechanism.
-- Overrides the newFileChannel(..) method to return new SecureFileChannel
implementation.
--- SecureFileChannel: overrides the truncate(..) method and delegate the
other calls to the original FileChannel implementation.
-- Overrides the remaining methods to delegate the calls to the original
FileSystemProvider.
Everything can be done without changing Ratis. Would this work for you?
(I was open to add a new File API and had filed RATIS-2585 for this JIRA.
But then learned that this JIRA might not be needed.)
> Pluggable file destruction hook for Raft log, snapshot, and storage cleanup
> ---------------------------------------------------------------------------
>
> Key: RATIS-2581
> URL: https://issues.apache.org/jira/browse/RATIS-2581
> Project: Ratis
> Issue Type: Improvement
> Components: RaftLog, snapshot
> Reporter: Haonan Hou
> Assignee: Haonan Hou
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ratis currently removes local storage files using normal filesystem operations
> such as Files.delete/deleteIfExists and FileChannel.truncate. This is fine for
> the default behavior, but some downstream projects embedding Ratis need to
> integrate with their own compliant file destruction mechanism.
> For example, a downstream system may need to destroy Raft log segments,
> snapshots, metadata temp files, and temporary snapshot installation files
> using
> a custom secure-delete / crypto-erase / audited destruction implementation
> instead of plain filesystem unlink/truncate.
> This request proposes adding a pluggable file destruction abstraction in
> Ratis,
> with the existing behavior as the default implementation.
> *Current Behavior*
> Ratis core cleanup paths eventually use normal delete/truncate operations.
> Examples include:
> - FileUtils.delete/deleteIfExists/deleteFully
> - FileUtils.truncateFile
> - raft log purge and truncate paths
> - snapshot installation cleanup
> - atomic metadata temp file cleanup
> - group remove storage directory cleanup
> There are also a few direct calls such as Files.delete,
> File.delete/deleteOnExit,
> or FileChannel.truncate that can bypass FileUtils.
> *Proposed Change*
> Introduce a small pluggable file destruction API, for example:
> {code:java}
> interface RatisFileDestroyer {
> void delete(Path path) throws IOException;
> void deleteIfExists(Path path) throws IOException;
> void truncate(Path path, long targetLength) throws IOException;
> } {code}
> The default implementation should preserve the current behavior:
> - Files.delete
> - Files.deleteIfExists
> - FileChannel.truncate
> Ratis should route its core local storage deletion/truncation paths through
> this
> abstraction, including:
> - recursive storage directory cleanup
> - Raft log segment purge
> - Raft log truncation
> - snapshot temp file cleanup
> - snapshot installation replacement cleanup
> - atomic output temp file abort cleanup
> - zero-size open log segment cleanup
> The API should be configurable by embedders, e.g. through RaftProperties,
> RaftServer builder configuration, ServiceLoader, or another Ratis-approved
> extension mechanism.
> *Compatibility*
> The default behavior remains unchanged.
> Existing users should not see behavior changes unless they explicitly
> configure
> a custom file destroyer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
