merlimat closed pull request #1172: Improve retention and expiry documentation 
(WIP)
URL: https://github.com/apache/incubator-pulsar/pull/1172
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/site/_sass/_syntax.scss b/site/_sass/_syntax.scss
index 45bc7b9f8..a41132249 100644
--- a/site/_sass/_syntax.scss
+++ b/site/_sass/_syntax.scss
@@ -149,6 +149,10 @@ p code {
     color: $sx-orange;
   }
 
+  .nv {
+    color: $sx-red;
+  }
+
   .na {
     color: $sx-7;
   }
diff --git a/site/docs/latest/getting-started/ConceptsAndArchitecture.md 
b/site/docs/latest/getting-started/ConceptsAndArchitecture.md
index 54b0dfbeb..045af3ff9 100644
--- a/site/docs/latest/getting-started/ConceptsAndArchitecture.md
+++ b/site/docs/latest/getting-started/ConceptsAndArchitecture.md
@@ -266,13 +266,35 @@ In BookKeeper, *journal* files contain BookKeeper 
transaction logs. Before makin
 
 A future version of BookKeeper will support *non-persistent messaging* and 
thus multiple durability modes at the topic level. This will enable you to set 
the durability mode at the topic level, replacing the `persistent` in topic 
names with a `non-persistent` indicator.
 
+## Message retention and expiry
+
+By default, Pulsar message {% popover brokers %}:
+
+* immediately delete *all* messages that have been {% popover acknowledged %} 
by a {% popover consumer %}, and
+* [persistently store](#persistent-storage) all unacknowledged messages in a 
message backlog.
+
+Pulsar has two features, however, that enable you to override this default 
behavior:
+
+* Message **retention** enables you to store messages that have been 
acknowledged by a consumer
+* Message **expiry** enables you to set a time to live (TTL) for messages that 
have not yet been acknowledged
+
+{% include admonition.html type="info" content='All message retention and 
expiry is managed at the [namespace](#namespaces) level. For a how-to, see the 
[Message retention and expiry](../../advanced/RetentionExpiry) admin 
documentation.' %}
+
+The diagram below illustrates both concepts:
+
+{% img /img/retention-expiry.png 80 %}
+
+With message retention, shown at the top, a <span style="color: 
#89b557;">retention policy</span> applied to all topics in a {% popover 
namespace %} dicates that some messages are durably stored in Pulsar even 
though they've already been acknowledged. Acknowledged messages that are not 
covered by the retention policy are <span style="color: 
#bb3b3e;">deleted</span>. Without a retention policy, *all* of the <span 
style="color: #19967d;">acknowledged messages</span> would be deleted.
+
+With message expiry, shown at the bottom, some messages are <span 
style="color: #bb3b3e;">deleted</span>, even though they <span style="color: 
#337db6;">haven't been acknowledged</span>, because they've expired according 
to the <span style="color: #e39441;">TTL applied to the namespace</span> (for 
example because a TTL of 5 minutes has been applied and the messages haven't 
been acknowledged but are 10 minutes old).
+
 ## Replication
 
 Pulsar enables messages to be produced and consumed in different 
geo-locations. For instance, your application may be publishing data in one 
region or market and you would like to process it for consumption in other 
regions or markets. [Geo-replication](../../admin/GeoReplication) in Pulsar 
enables you to do that.
 
 ## Multi-tenancy
 
-Pulsar was created from the ground up as a {% popover multi-tenant %} system. 
To support multi-tenancy, Pulsar has a concept of {% popover properties %}. 
Properties can be spread across {% popover clusters %} and can each have their 
own [authentication and authorization](../../admin/Authz) scheme applied to 
them. They are also the administrative unit at which [storage quotas](TODO), 
[message TTL](TODO), and [isolation policies](TODO) can be managed.
+Pulsar was created from the ground up as a {% popover multi-tenant %} system. 
To support multi-tenancy, Pulsar has a concept of {% popover properties %}. 
Properties can be spread across {% popover clusters %} and can each have their 
own [authentication and authorization](../../admin/Authz) scheme applied to 
them. They are also the administrative unit at which [storage quotas](TODO), 
[message TTL](../../advanced/RetentionExpiry#time-to-live-ttl), and isolation 
policies can be managed.
 
 The multi-tenant nature of Pulsar is reflected mostly visibly in topic URLs, 
which have this structure:
 
diff --git a/site/img/retention-expiry.png b/site/img/retention-expiry.png
new file mode 100644
index 000000000..5b9f5fd94
Binary files /dev/null and b/site/img/retention-expiry.png differ


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to