[ https://issues.apache.org/jira/browse/NIFI-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906313#comment-16906313 ]
Brandon DeVries commented on NIFI-4775: --------------------------------------- I don't see any reason not to update the rocksdb version. I'll rerun my tests to confirm, but I don't anticipate a problem. And I'm working on updating the docs now. > Create a FlowFile repo backed by RocksDB > ---------------------------------------- > > Key: NIFI-4775 > URL: https://issues.apache.org/jira/browse/NIFI-4775 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework > Reporter: Mark Payne > Assignee: Brandon DeVries > Priority: Major > Fix For: 1.10.0 > > Attachments: RocksDBFlowFileRepo.html, rocksdb-flowfile-repo.adoc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently, when a FlowFile is written to the FlowFile Repository, the repo > can either fsync or not, depending on nifi.properties. We should allow a > third option, of fsync only for CREATE events. In this case, if we receive > new data from a source we can fsync the update to the FlowFile Repository > before ACK'ing the data from the source. This allows us to guarantee data > persistence without the overhead of an fsync for every FlowFile Repository > update. > It may make sense, though, to be a bit more selective about when do this. For > example if the source is a system that does not allow us to acknowledge the > receipt of data, such as a ListenUDP processor, this doesn't really buy us > much. In such a case, we could be smart about avoiding the high cost of an > fsync. However, for something like GetSFTP where we have to remove the file > in order to 'acknowledge receipt' we can ensure that we wait for the fsync > before proceeding. > NOTE: This functionality was ultimately provided in a new implementation > backed by RocksDB > -- This message was sent by Atlassian JIRA (v7.6.14#76016)