This is an automated email from the ASF dual-hosted git repository.
huanlimeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new b742c96 [docs] [ISSUE 11625] Update the intro of Messaging doc
(#11667)
b742c96 is described below
commit b742c96c88c87abb8dd234549cc7cf485cec6e98
Author: sijia-w <[email protected]>
AuthorDate: Wed Aug 18 02:39:06 2021 +0200
[docs] [ISSUE 11625] Update the intro of Messaging doc (#11667)
* update desc
* update
---
site2/docs/concepts-messaging.md | 62 +++++++++++++++++++++-------------------
1 file changed, 32 insertions(+), 30 deletions(-)
diff --git a/site2/docs/concepts-messaging.md b/site2/docs/concepts-messaging.md
index c46eb4f..2d21b40 100644
--- a/site2/docs/concepts-messaging.md
+++ b/site2/docs/concepts-messaging.md
@@ -4,9 +4,11 @@ title: Messaging
sidebar_label: Messaging
---
-Pulsar is built on the
[publish-subscribe](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern)
pattern (often abbreviated to pub-sub). In this pattern,
[producers](#producers) publish messages to [topics](#topics).
[Consumers](#consumers) [subscribe](#subscription-modes) to those topics,
process incoming messages, and send an acknowledgement when processing is
complete.
+Pulsar is built on the
[publish-subscribe](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern)
pattern (often abbreviated to pub-sub). In this pattern,
[producers](#producers) publish messages to [topics](#topics);
[consumers](#consumers) [subscribe](#subscription-modes) to those topics,
process incoming messages, and send [acknowledgements](#acknowledgement) to the
broker when processing is finished.
-When a subscription is created, Pulsar
[retains](concepts-architecture-overview.md#persistent-storage) all messages,
even if the consumer is disconnected. Retained messages are discarded only when
a consumer acknowledges that those messages are processed successfully.
+When a subscription is created, Pulsar
[retains](concepts-architecture-overview.md#persistent-storage) all messages,
even if the consumer is disconnected. The retained messages are discarded only
when a consumer acknowledges that all these messages are processed
successfully.
+
+If the consumption of a message fails and you want this message to be consumed
again, then you can enable the automatic redelivery of this message by sending
a [negative acknowledgement](negative-acknowledgement) to the broker or
enabling the [acknowledgement timeout](acknowledgement-timeout) for
unacknowledged messages.
## Messages
@@ -38,11 +40,11 @@ The default size of a message is 5 MB. You can configure
the max size of a messa
# The max size of the netty frame (in bytes). Any messages received larger
than this value are rejected. The default value is 5 MB.
nettyMaxFrameSizeBytes=5253120
```
-> For more information on Pulsar message contents, see Pulsar [binary
protocol](developing-binary-protocol.md).
+> For more information on Pulsar messages, see Pulsar [binary
protocol](developing-binary-protocol.md).
## Producers
-A producer is a process that attaches to a topic and publishes messages to a
Pulsar [broker](reference-terminology.md#broker). The Pulsar broker process the
messages.
+A producer is a process that attaches to a topic and publishes messages to a
Pulsar [broker](reference-terminology.md#broker). The Pulsar broker processes
the messages.
### Send modes
@@ -65,11 +67,11 @@ You can have different types of access modes on topics for
producers.
> **Note**
>
-> Once an application creates a producer with the `Exclusive` or
`WaitForExclusive` access mode successfully, the instance of the application is
guaranteed to be the **only one writer** on the topic. Other producers trying
to produce on this topic get errors immediately or have to wait until they get
the `Exclusive` access.
+> Once an application creates a producer with `Exclusive` or
`WaitForExclusive` access mode successfully, the instance of this application
is guaranteed to be the **only writer** to the topic. Any other producers
trying to produce messages on this topic will either get errors immediately or
have to wait until they get the `Exclusive` access.
>
> For more information, see [PIP 68: Exclusive
> Producer](https://github.com/apache/pulsar/wiki/PIP-68:-Exclusive-Producer).
-You can set producer access mode through Java Client API. For more
information, see `ProducerAccessMode` in
[ProducerBuilder.java](https://github.com/apache/pulsar/blob/fc5768ca3bbf92815d142fe30e6bfad70a1b4fc6/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java).
+You can set producer access mode through Java Client API. For more
information, see `ProducerAccessMode` in
[ProducerBuilder.java](https://github.com/apache/pulsar/blob/fc5768ca3bbf92815d142fe30e6bfad70a1b4fc6/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java)
file.
### Compression
@@ -87,14 +89,14 @@ When batching is enabled, the producer accumulates and
sends a batch of messages
In Pulsar, batches are tracked and stored as single units rather than as
individual messages. Consumer unbundles a batch into individual messages.
However, scheduled messages (configured through the `deliverAt` or the
`deliverAfter` parameter) are always sent as individual messages even batching
is enabled.
-In general, a batch is acknowledged when all of its messages are acknowledged
by a consumer. It means unexpected failures, negative acknowledgements, or
acknowledgement timeouts can result in redelivery of all messages in a batch,
even if some of the messages are acknowledged.
+In general, a batch is acknowledged when all of its messages are acknowledged
by a consumer. It means that when **not all** batch messages are acknowledged,
then unexpected failures, negative acknowledgements, or acknowledgement
timeouts can result in a redelivery of all messages in this batch.
-To avoid redelivering acknowledged messages in a batch to the consumer, Pulsar
introduces batch index acknowledgement since Pulsar 2.6.0. When batch index
acknowledgement is enabled, the consumer filters out the batch index that has
been acknowledged and sends the batch index acknowledgement request to the
broker. The broker maintains the batch index acknowledgement status and tracks
the acknowledgement status of each batch index to avoid dispatching
acknowledged messages to the consumer [...]
+To avoid redelivering acknowledged messages in a batch to the consumer, Pulsar
introduces batch index acknowledgement since Pulsar 2.6.0. When batch index
acknowledgement is enabled, the consumer filters out the batch index that has
been acknowledged and sends the batch index acknowledgement request to the
broker. The broker maintains the batch index acknowledgement status and tracks
the acknowledgement status of each batch index to avoid dispatching
acknowledged messages to the consumer [...]
By default, batch index acknowledgement is disabled
(`acknowledgmentAtBatchIndexLevelEnabled=false`). You can enable batch index
acknowledgement by setting the `acknowledgmentAtBatchIndexLevelEnabled`
parameter to `true` at the broker side. Enabling batch index acknowledgement
results in more memory overheads.
### Chunking
-When you enable chunking, read the following instructions.
+Before you enable chunking, read the following instructions.
- Batching and chunking cannot be enabled simultaneously. To enable chunking,
you must disable batching in advance.
- Chunking is only supported for persisted topics.
- Chunking is only supported for the exclusive and failover subscription modes.
@@ -138,14 +140,14 @@ Client libraries provide listener implementation for
consumers. For example, the
### Acknowledgement
-When a consumer consumes a message successfully, the consumer sends an
acknowledgement request to the broker. This message is permanently stored, and
then deleted only after all the subscriptions have acknowledged it. If you want
to store the message that has been acknowledged by a consumer, you need to
configure the [message retention
policy](concepts-messaging.md#message-retention-and-expiry).
+The consumer sends an acknowledgement request to the broker after it consumes
a message successfully. Then, this consumed message will be permanently stored,
and be deleted only after all the subscriptions have acknowledged it. If you
want to store the messages that have been acknowledged by a consumer, you need
to configure the [message retention
policy](concepts-messaging.md#message-retention-and-expiry).
-For a batch message, if batch index acknowledgement is enabled, the broker
maintains the batch index acknowledgement status and tracks the acknowledgement
status of each batch index to avoid dispatching acknowledged messages to the
consumer. When all indexes of the batch message are acknowledged, the batch
message is deleted. For details about the batch index acknowledgement, see
[batching](#batching).
+For batch messages, you can enable batch index acknowledgement to avoid
dispatching acknowledged messages to the consumer. For details about batch
index acknowledgement, see [batching](#batching).
-Messages can be acknowledged in the following two ways:
+Messages can be acknowledged in one of the following two ways:
-- Messages are acknowledged individually. With individual acknowledgement, the
consumer needs to acknowledge each message and sends an acknowledgement request
to the broker.
-- Messages are acknowledged cumulatively. With cumulative acknowledgement, the
consumer only needs to acknowledge the last message it received. All messages
in the stream up to (and including) the provided message are not re-delivered
to that consumer.
+- Being acknowledged individually. With individual acknowledgement, the
consumer acknowledges each message and sends an acknowledgement request to the
broker.
+- Being acknowledged cumulatively. With cumulative acknowledgement, the
consumer **only** acknowledges the last message it received. All messages in
the stream up to (and including) the provided message are not redelivered to
that consumer.
If you want to acknowledge messages individually, you can use the following
API.
@@ -158,20 +160,20 @@ consumer.acknowledgeCumulative(msg);
```
-> **Note**
+> **Note**
> Cumulative acknowledgement cannot be used in the [shared subscription
> mode](#subscription-modes), because the shared subscription mode involves
> multiple consumers who have access to the same subscription. In the shared
> subscription mode, messages are acknowledged individually.
### Negative acknowledgement
-When a consumer does not consume a message successfully at a time, and wants
to consume the message again, the consumer sends a negative acknowledgement to
the broker, and then the broker redelivers the message.
+When a consumer fails to consume a message and intends to consume it again,
this consumer should send a negative acknowledgement to the broker. Then, the
broker will redeliver this message to the consumer.
-Messages are negatively acknowledged one by one or cumulatively, which depends
on the consumption subscription mode.
+Messages are negatively acknowledged individually or cumulatively, depending
on the consumption subscription mode.
In the exclusive and failover subscription modes, consumers only negatively
acknowledge the last message they receive.
-In the shared and Key_Shared subscription modes, you can negatively
acknowledge messages individually.
+In the shared and Key_Shared subscription modes, consumers can negatively
acknowledge messages individually.
-Be aware that negative acknowledgment on ordered subscription types, such as
Exclusive, Failover and Key_Shared, can cause failed messages to arrive
consumers out of the original order.
+Be aware that negative acknowledgments on ordered subscription types, such as
Exclusive, Failover and Key_Shared, might cause failed messages being sent to
consumers out of the original order.
If you want to acknowledge messages negatively, you can use the following API.
@@ -180,18 +182,18 @@ If you want to acknowledge messages negatively, you can
use the following API.
consumer.negativeAcknowledge(msg);
```
-> **Note**
-> If batching is enabled, other messages and the negatively acknowledged
messages in the same batch are redelivered to the consumer.
+> **Note**
+> If batching is enabled, all messages in one batch are redelivered to the
consumer.
### Acknowledgement timeout
-If a message is not consumed successfully, and you want to trigger the broker
to redeliver the message automatically, you can adopt the unacknowledged
message automatic re-delivery mechanism. Client tracks the unacknowledged
messages within the entire `acktimeout` time range, and sends a `redeliver
unacknowledged messages` request to the broker automatically when the
acknowledgement timeout is specified.
+If a message is not consumed successfully, and you want the broker to
redeliver this message automatically, then you can enable automatic redelivery
mechanism for unacknowledged messages. With automatic redelivery enabled, the
client tracks the unacknowledged messages within the entire `acktimeout` time
range, and sends a `redeliver unacknowledged messages` request to the broker
automatically when the acknowledgement timeout is specified.
-> **Note**
-> If batching is enabled, other messages and the unacknowledged messages in
the same batch are redelivered to the consumer.
+> **Note**
+> If batching is enabled, all messages in one batch are redelivered to the
consumer.
> **Note**
-> Prefer negative acknowledgements over acknowledgement timeout. Negative
acknowledgement controls the re-delivery of individual messages with more
precision, and avoids invalid redeliveries when the message processing time
exceeds the acknowledgement timeout.
+> The negative acknowledgement is preferable over the acknowledgement timeout
since negative acknowledgement controls the redelivery of individual messages
more precisely and avoids invalid redeliveries when the message processing time
exceeds the acknowledgement timeout.
### Dead letter topic
@@ -230,7 +232,7 @@ Consumer<byte[]> consumer =
pulsarClient.newConsumer(Schema.BYTES)
```
-Dead letter topic depends on message re-delivery. Messages are redelivered
either due to [acknowledgement timeout](#acknowledgement-timeout) or [negative
acknowledgement](#negative-acknowledgement). If you are going to use negative
acknowledgement on a message, make sure it is negatively acknowledged before
the acknowledgement timeout.
+Dead letter topic depends on message redelivery. Messages are redelivered
either due to [acknowledgement timeout](#acknowledgement-timeout) or [negative
acknowledgement](#negative-acknowledgement). If you are going to use negative
acknowledgement on a message, make sure it is negatively acknowledged before
the acknowledgement timeout.
> **Note**
> Currently, dead letter topic is enabled in the Shared and Key_Shared
> subscription modes.
@@ -281,7 +283,7 @@ Topic name component | Description
`namespace` | The administrative unit of the topic, which acts as a
grouping mechanism for related topics. Most topic configuration is performed at
the [namespace](#namespaces) level. Each tenant has one or multiple namespaces.
`topic` | The final part of the name. Topic names have no special
meaning in a Pulsar instance.
-> **No need to explicitly create new topics**
+> **No need to explicitly create new topics**
> You do not need to explicitly create topics in Pulsar. If a client attempts
> to write or receive messages to/from a topic that does not yet exist, Pulsar
> creates that topic under the namespace provided in the [topic name](#topics)
> automatically.
> If no tenant or namespace is specified when a client creates a topic, the
> topic is created in the default tenant and namespace. You can also create a
> topic in a specified tenant and namespace, such as
> `persistent://my-tenant/my-namespace/my-topic`.
> `persistent://my-tenant/my-namespace/my-topic` means the `my-topic` topic is
> created in the `my-namespace` namespace of the `my-tenant` tenant.
@@ -332,7 +334,7 @@ In *shared* or *round robin* mode, multiple consumers can
attach to the same sub
In the diagram below, **Consumer-C-1** and **Consumer-C-2** are able to
subscribe to the topic, but **Consumer-C-3** and others could as well.
-> **Limitations of shared mode**
+> **Limitations of shared mode**
> When using shared mode, be aware that:
> * Message ordering is not guaranteed.
> * You cannot use cumulative acknowledgment with shared mode.
@@ -343,7 +345,7 @@ In the diagram below, **Consumer-C-1** and **Consumer-C-2**
are able to subscrib
In *Key_Shared* mode, multiple consumers can attach to the same subscription.
Messages are delivered in a distribution across consumers and message with same
key or same ordering key are delivered to only one consumer. No matter how many
times the message is re-delivered, it is delivered to the same consumer. When a
consumer connected or disconnected will cause served consumer change for some
key of message.
-> **Limitations of Key_Shared mode**
+> **Limitations of Key_Shared mode**
> When you use Key_Shared mode, be aware that:
> * You need to specify a key or orderingKey for messages.
> * You cannot use cumulative acknowledgment with Key_Shared mode.
@@ -364,7 +366,7 @@ When a consumer subscribes to a Pulsar topic, by default it
subscribes to one sp
When subscribing to multiple topics, the Pulsar client automatically makes a
call to the Pulsar API to discover the topics that match the regex
pattern/list, and then subscribe to all of them. If any of the topics do not
exist, the consumer auto-subscribes to them once the topics are created.
-> **No ordering guarantees across multiple topics**
+> **No ordering guarantees across multiple topics**
> When a producer sends messages to a single topic, all messages are
> guaranteed to be read from that topic in the same order. However, these
> guarantees do not hold across multiple topics. So when a producer sends
> message to multiple topics, the order in which messages are read from those
> topics is not guaranteed to be the same.
The following are multi-topic subscription examples for Java.
