[
https://issues.apache.org/jira/browse/NIFI-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16371874#comment-16371874
]
ASF GitHub Bot commented on NIFI-4775:
--------------------------------------
Github user devriesb commented on the issue:
https://github.com/apache/nifi/pull/2416
I'll grant NIFI-4775 may raise issues with my proposed solution. However,
there is a problem right now. My proposed solution addresses the problem right
now. Future modification may require adjustments to previous assumptions.
That, however, is a problem for the future.
In any case, after doing some experimentation, I'm not sure the current
version of NIFI-4775 is the correct approach. And whatever the eventual
approach is, it may more appropriately be a new implementation (as discussed
above). I don't think we should put off correcting current bugs because they
may complicate potential future features.
> Allow FlowFile Repository to optionally perform fsync when writing CREATE
> events but not other events
> -----------------------------------------------------------------------------------------------------
>
> Key: NIFI-4775
> URL: https://issues.apache.org/jira/browse/NIFI-4775
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Mark Payne
> Priority: Major
>
> Currently, when a FlowFile is written to the FlowFile Repository, the repo
> can either fsync or not, depending on nifi.properties. We should allow a
> third option, of fsync only for CREATE events. In this case, if we receive
> new data from a source we can fsync the update to the FlowFile Repository
> before ACK'ing the data from the source. This allows us to guarantee data
> persistence without the overhead of an fsync for every FlowFile Repository
> update.
> It may make sense, though, to be a bit more selective about when do this. For
> example if the source is a system that does not allow us to acknowledge the
> receipt of data, such as a ListenUDP processor, this doesn't really buy us
> much. In such a case, we could be smart about avoiding the high cost of an
> fsync. However, for something like GetSFTP where we have to remove the file
> in order to 'acknowledge receipt' we can ensure that we wait for the fsync
> before proceeding.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
