[
https://issues.apache.org/jira/browse/NIFI-13546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17866813#comment-17866813
]
ASF subversion and git services commented on NIFI-13546:
--------------------------------------------------------
Commit 0e7e511aec1ad2c4e602cf1ef789c52edf2cac7e in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0e7e511aec ]
NIFI-13546 Fixed Content Repository recovery when archive directories are
missing
Ensure on startup that Content Repo archive directories are created, if
configured to do so, rather than creating them on demand. If IOException thrown
when archiving file, delete it instead. Cleaned up some code duplication
between remove(ResourceClaim) and archive(ResourceClaim) methods
This closes #9079
Signed-off-by: David Handermann <[email protected]>
> When Content Repo completely runs out of disk space, it fails to recover if
> archive directories not created
> -----------------------------------------------------------------------------------------------------------
>
> Key: NIFI-13546
> URL: https://issues.apache.org/jira/browse/NIFI-13546
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Fix For: 2.0.0-M5
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When filling up the content repo I see the following error:
> {code:java}
> 2024-07-10 19:03:06,841 WARN [FileSystemRepository Workers Thread-3]
> org.apache.nifi.controller.repository.FileSystemRepository Failed to archive
> StandardResourceClaim[id=1720629393533-2, container=repo1, section=2]
> java.nio.file.FileSystemException: /nifi/content_repository_1/2/archive: No
> space left on device
> at
> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
> at
> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
> at
> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> at
> java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:438)
> at java.base/java.nio.file.Files.createDirectory(Files.java:699)
> at
> java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
> at java.base/java.nio.file.Files.createDirectories(Files.java:793)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.archive(FileSystemRepository.java:1199)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.archive(FileSystemRepository.java:1158)
> at
> org.apache.nifi.controller.repository.FileSystemRepository$ArchiveOrDestroyDestructableClaims.run(FileSystemRepository.java:1464)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583){code}
> Because it cannot create the 'archive' directory, the file doesn't get moved
> to archive. As a result, it never gets purged from the system, even after
> emptying all FlowFile queues.
> Additionally, it creates 10's of thousands of files that are 0 bytes and then
> fails to write to them, and since they can't be archived they just stay
> around.
> In the case that we encounter an IOException when attempting to move a file
> into archive, we should generate a WARN log and delete it, rather than simply
> moving on.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
