This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
     new 2f1ef0b  Improve retention and expiry documentation (WIP) (#1172)
2f1ef0b is described below

commit 2f1ef0b8fff120224fb00fb28134905b8185a3b8
Author: Luc Perkins <lucperk...@gmail.com>
AuthorDate: Fri Feb 23 15:23:53 2018 -0800

    Improve retention and expiry documentation (WIP) (#1172)
    
    * add retention/expiry diagram
    
    * add color for $ in bash snippets
    
    * modify colors in diagram
    
    * alter diagram colors once again
---
 site/_sass/_syntax.scss                            |   4 ++++
 .../getting-started/ConceptsAndArchitecture.md     |  24 ++++++++++++++++++++-
 site/img/retention-expiry.png                      | Bin 0 -> 113741 bytes
 3 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/site/_sass/_syntax.scss b/site/_sass/_syntax.scss
index 45bc7b9..a411322 100644
--- a/site/_sass/_syntax.scss
+++ b/site/_sass/_syntax.scss
@@ -149,6 +149,10 @@ p code {
     color: $sx-orange;
   }
 
+  .nv {
+    color: $sx-red;
+  }
+
   .na {
     color: $sx-7;
   }
diff --git a/site/docs/latest/getting-started/ConceptsAndArchitecture.md 
b/site/docs/latest/getting-started/ConceptsAndArchitecture.md
index 54b0dfb..045af3f 100644
--- a/site/docs/latest/getting-started/ConceptsAndArchitecture.md
+++ b/site/docs/latest/getting-started/ConceptsAndArchitecture.md
@@ -266,13 +266,35 @@ In BookKeeper, *journal* files contain BookKeeper 
transaction logs. Before makin
 
 A future version of BookKeeper will support *non-persistent messaging* and 
thus multiple durability modes at the topic level. This will enable you to set 
the durability mode at the topic level, replacing the `persistent` in topic 
names with a `non-persistent` indicator.
 
+## Message retention and expiry
+
+By default, Pulsar message {% popover brokers %}:
+
+* immediately delete *all* messages that have been {% popover acknowledged %} 
by a {% popover consumer %}, and
+* [persistently store](#persistent-storage) all unacknowledged messages in a 
message backlog.
+
+Pulsar has two features, however, that enable you to override this default 
behavior:
+
+* Message **retention** enables you to store messages that have been 
acknowledged by a consumer
+* Message **expiry** enables you to set a time to live (TTL) for messages that 
have not yet been acknowledged
+
+{% include admonition.html type="info" content='All message retention and 
expiry is managed at the [namespace](#namespaces) level. For a how-to, see the 
[Message retention and expiry](../../advanced/RetentionExpiry) admin 
documentation.' %}
+
+The diagram below illustrates both concepts:
+
+{% img /img/retention-expiry.png 80 %}
+
+With message retention, shown at the top, a <span style="color: 
#89b557;">retention policy</span> applied to all topics in a {% popover 
namespace %} dicates that some messages are durably stored in Pulsar even 
though they've already been acknowledged. Acknowledged messages that are not 
covered by the retention policy are <span style="color: 
#bb3b3e;">deleted</span>. Without a retention policy, *all* of the <span 
style="color: #19967d;">acknowledged messages</span> would be deleted.
+
+With message expiry, shown at the bottom, some messages are <span 
style="color: #bb3b3e;">deleted</span>, even though they <span style="color: 
#337db6;">haven't been acknowledged</span>, because they've expired according 
to the <span style="color: #e39441;">TTL applied to the namespace</span> (for 
example because a TTL of 5 minutes has been applied and the messages haven't 
been acknowledged but are 10 minutes old).
+
 ## Replication
 
 Pulsar enables messages to be produced and consumed in different 
geo-locations. For instance, your application may be publishing data in one 
region or market and you would like to process it for consumption in other 
regions or markets. [Geo-replication](../../admin/GeoReplication) in Pulsar 
enables you to do that.
 
 ## Multi-tenancy
 
-Pulsar was created from the ground up as a {% popover multi-tenant %} system. 
To support multi-tenancy, Pulsar has a concept of {% popover properties %}. 
Properties can be spread across {% popover clusters %} and can each have their 
own [authentication and authorization](../../admin/Authz) scheme applied to 
them. They are also the administrative unit at which [storage quotas](TODO), 
[message TTL](TODO), and [isolation policies](TODO) can be managed.
+Pulsar was created from the ground up as a {% popover multi-tenant %} system. 
To support multi-tenancy, Pulsar has a concept of {% popover properties %}. 
Properties can be spread across {% popover clusters %} and can each have their 
own [authentication and authorization](../../admin/Authz) scheme applied to 
them. They are also the administrative unit at which [storage quotas](TODO), 
[message TTL](../../advanced/RetentionExpiry#time-to-live-ttl), and isolation 
policies can be managed.
 
 The multi-tenant nature of Pulsar is reflected mostly visibly in topic URLs, 
which have this structure:
 
diff --git a/site/img/retention-expiry.png b/site/img/retention-expiry.png
new file mode 100644
index 0000000..5b9f5fd
Binary files /dev/null and b/site/img/retention-expiry.png differ

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

Reply via email to