This is an automated email from the ASF dual-hosted git repository.
clebertsuconic pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/activemq-artemis.git
The following commit(s) were added to refs/heads/main by this push:
new 2efc4967b6 ARTEMIS-4495 Improving data retention documentation
2efc4967b6 is described below
commit 2efc4967b63197ea05beada291cdbc1a5d0dc41b
Author: Clebert Suconic <[email protected]>
AuthorDate: Thu Nov 9 14:36:23 2023 -0500
ARTEMIS-4495 Improving data retention documentation
---
.../api/core/management/ActiveMQServerControl.java | 16 ++++++
docs/user-manual/data-retention.adoc | 54 +++++++++++++++++++++
docs/user-manual/images/replay-method.png | Bin 0 -> 206230 bytes
docs/user-manual/persistence.adoc | 46 +-----------------
4 files changed, 72 insertions(+), 44 deletions(-)
diff --git
a/artemis-core-client/src/main/java/org/apache/activemq/artemis/api/core/management/ActiveMQServerControl.java
b/artemis-core-client/src/main/java/org/apache/activemq/artemis/api/core/management/ActiveMQServerControl.java
index 84e2a9cefb..9e4e00f596 100644
---
a/artemis-core-client/src/main/java/org/apache/activemq/artemis/api/core/management/ActiveMQServerControl.java
+++
b/artemis-core-client/src/main/java/org/apache/activemq/artemis/api/core/management/ActiveMQServerControl.java
@@ -2000,11 +2000,27 @@ public interface ActiveMQServerControl {
@Operation(desc = "forces the broker to reload its configuration file",
impact = MBeanOperationInfo.ACTION)
void reloadConfigurationFile() throws Exception;
+ /**
+ * Replays messages from all files in the retention folder that match an
address and filter.
+ * @param address
+ * @param target
+ * @param filter
+ * @throws Exception
+ */
@Operation(desc = "Replays messages from all files in the retention folder
that match an address and filter.", impact = MBeanOperationInfo.ACTION)
void replay(@Parameter(name = "address", desc = "Name of the address to
replay") String address,
@Parameter(name = "target", desc = "Where the replay data
should be sent") String target,
@Parameter(name = "filter", desc = "Filter to apply on message
selection. Null means everything matching the address") String filter) throws
Exception;
+ /**
+ * Replays messages from a configurable subset of the files in the
retention folder that match an address and filter.
+ * @param startScan
+ * @param endScan
+ * @param address
+ * @param target
+ * @param filter
+ * @throws Exception
+ */
@Operation(desc = "Replays messages from a configurable subset of the files
in the retention folder that match an address and filter.", impact =
MBeanOperationInfo.ACTION)
void replay(@Parameter(name = "startScanDate", desc = "Start date where we
will start scanning for journals to replay. Format YYYYMMDDHHMMSS") String
startScan,
@Parameter(name = "endScanDate", desc = "Finish date where we
will stop scannning for journals to replay. Format YYYYMMDDHHMMSS") String
endScan,
diff --git a/docs/user-manual/data-retention.adoc
b/docs/user-manual/data-retention.adoc
new file mode 100644
index 0000000000..c55750ec64
--- /dev/null
+++ b/docs/user-manual/data-retention.adoc
@@ -0,0 +1,54 @@
+= Data Retention
+:idprefix:
+:idseparator: -
+
+If you enable `journal-retention` on broker.xml, ActiveMQ Artemis will keep
copy of every data that has passed through the broker on this folder.
+
+[,xml]
+----
+ ...
+ <journal-retention unit="DAYS" directory="history" period="365"
storage-limit="10G"/>
+ ...
+----
+
+ActiveMQ Artemis will keep a copy of each generated journal file, up to the
configured retention period, at the unit chose.
+On the example above the system would keep all the journal files up to 365
days.
+
+It is also possible to limit the number of files kept on the retention
directory.
+You can keep a storage-limit, and the system will start removing older files
when you have more files than the configured storage limit.
+
+Notice the storage limit is optional however you need to be careful to not run
out of disk space at the retention folder or the broker might be shutdown
because of a critical IO failure.
+
+You can use the CLI tools to inspect and recover data from the history, by
just passing the journal folder being the retention directory.
+
+Example:
+
+[,shell]
+----
+./artemis data print --journal ../data/history
+----
+
+==== Paging and Large Messages
+When retention is enabled, paging and large messages are also stored on the
retention folder for replaying, so replay will also work for both paged and
large messages.
+
+==== Non Persistent Messages
+Non persistent messages are never stored anywhere in the broker. If you intend
to retain data your producers should be using Durable messages.
+
+== Recovering data from retention
+
+To recover data from the retention folder, you can use the method
link:https://activemq.apache.org/components/artemis/documentation/javadocs/javadoc-latest/org/apache/activemq/artemis/api/core/management/ActiveMQServerControl.html#replay(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String)[replay]
on the management console:
+
+image::images/replay-method.png[align="center"]
+
+The broker will scan the set of files accordingly to the parameters you passed
and it will send these methods to a target queue that you selected.
+
+=== CLI recover operation
+
+The CLI recover operation is intended as a low level operation, where data is
read and recovered directly into a set of journal files.
+Notice uou should never manipulate journal files that are being used by a live
broker as that could risk the integrity of your data files and running broker.
+
+[,shell]
+----
+./artemis data recovery --journal ../data/history --target ../data/recovered
--large-messages ../data/large-messages
+----
+
diff --git a/docs/user-manual/images/replay-method.png
b/docs/user-manual/images/replay-method.png
new file mode 100644
index 0000000000..918849941f
Binary files /dev/null and b/docs/user-manual/images/replay-method.png differ
diff --git a/docs/user-manual/persistence.adoc
b/docs/user-manual/persistence.adoc
index d1c489ca8c..b4715dd24d 100644
--- a/docs/user-manual/persistence.adoc
+++ b/docs/user-manual/persistence.adoc
@@ -29,51 +29,9 @@ The journal also fully supports transactional operation if
required, supporting
The majority of the journal is written in Java, however we abstract out the
interaction with the actual file system to allow different pluggable
implementations.
Apache ActiveMQ Artemis ships with two implementations:
-=== Journal Retention
+=== Journal and Data Retention
-If you enable `journal-retention` on broker.xml, ActiveMQ Artemis will keep
copy of every data that has passed through the broker on this folder.
-
-[,xml]
-----
- ...
- <journal-retention unit="DAYS" directory="history" period="365"
storage-limit="10G"/>
- ...
-----
-
-ActiveMQ Artemis will keep a copy of each generated journal file, up to the
configured retention period, at the unit chose.
-On the example above the system would keep all the journal files up to 365
days.
-
-It is also possible to limit the number of files kept on the retention
directory.
-You can keep a storage-limit, and the system will start removing older files
when you have more files than the configured storage limit.
-
-Notice the storage limit is optional however you need to be careful to not run
out of disk space at the retention folder or the broker might be shutdown
because of a critical IO failure.
-
-You can use the CLI tools to inspect and recover data from the history, by
just passing the journal folder being the retention directory.
-
-Example:
-
-[,shell]
-----
-./artemis data print --journal ../data/history
-----
-
-To recover the messages from the history:
-
-[,shell]
-----
-./artemis data recovery --journal ../data/history --target ../data/recovered
--large-messages ../data/large-messages
-----
-
-It is important that you don't call recover into a the journal while the
broker is alive.
-As a matter of fact the current recommendations is to do that on a new journal
directory.
-Perhaps on a new broker so you can inspect and transfer these messages.
-
-The retention feature is in its current form very simple and intended for
emergency situations.
-If you think it is useful new options to recover the data could be added,
perhaps thorugh the admin console and other possibilities.
-Please share your feedback on this area, and as always Pull Requests are
welcomed!
-
-Also the recovery CLI tool will recover every data on the selected folder.
-It is important that you do some maintenance and copy the files and interval
you need to a new location before you call recover.
+ActiveMQ Artemis provides a way to store and replay historical data. Refer to
xref:data-retention.adoc[Data Retention] chapter for more information.
=== Java https://en.wikipedia.org/wiki/New_I/O[NIO]
