lordgamez commented on code in PR #1713:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1713#discussion_r1447342775
##########
CONFIGURE.md:
##########
@@ -287,6 +341,60 @@ Another option to define a state storage is to use the
following properties in t
nifi.state.storage.local.auto.persistence.interval=0 sec
+### Configuring Repositories
+
+Apache MiNiFi C++ uses three repositories similarily to Apache NiFi:
+- The Flow File Repository holds the metadata (like flow file attributes) of
the flow Files.
+- The Content Repository holds the content of the flow files.
+- The Provenance Repository holds the history of the flow files.
+
+The underlying implementation to use for these repositories can be configured
in the minifi.properties file.
+
+The Flow File Repository can be configured with the
`nifi.flowfile.repository.class.name` property. If not specified, it uses the
`FlowFileRepository` class by default, which stores the flow file metadata in a
RocksDB database. Alternatively it can be configured to use a
`VolatileFlowFileRepository` that keeps the state in memory and flushes upon
restart, or the `NoOpRepository` for not keeping any state.
+
+ # in minifi.properties
+ nifi.flowfile.repository.class.name=VolatileFlowFileRepository
+
+The Content Repository can be configured with the
`nifi.content.repository.class.name` property. If not specified, it uses the
`DatabaseContentRepository` class by default, which persists the content in a
RocksDB database. `DatabaseContentRepository` is also the default value
specified in the minifi.properties file. Alternatively it can be configured to
use a `VolatileContentRepository` that keeps the state in memory and flushes
upon restart, or the `FileSystemRepository` to keep the state in regular files.
+
+**NOTE:** RocksDB database has a limit of 4GB for the size of a database
object. Due to this if you are expecting to process larger flow files than 4GB
you should use the `FileSystemRepository`. The downside of using
`FileSystemRepository` is that it does not have the transactional guarantees of
the RocksDB repository implementation.
+
+ # in minifi.properties
+ nifi.content.repository.class.name=FileSystemRepository
+
+The Provenance Repository can be configured with the
`nifi.provenance.repository.class.name` property. If not specified, it uses the
`ProvenanceRepository` class by default, which persists the provenance events
in a RocksDB database. Alternatively it can be configured to use a
`VolatileProvenanceRepository` that keeps the state in memory and flushes upon
restart, or the `NoOpRepository` to not keep track of the provenance events. By
default we do not keep track of the provenance data, so `NoOpRepository` is the
value specified in the default minifi.properties file.
+
+ # in minifi.properties
+ nifi.flowfile.repository.class.name=NoOpRepository
+
+
+### Configuring Volatile Repositories
+As stated before each of the repositories can be configured to be volatile
(state kept in memory and flushed upon restart) or persistent. Volatile
repositories have some additional options, that can be specified in the
following ways:
Review Comment:
In this case the repository write replaces the oldest entry in round robin
manner, so the oldest entries will be lost. I added a note about this in
999a2039a9457abf91dae7af2f75a5910d7ce86f
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
