This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.wiki.git
The following commit(s) were added to refs/heads/master by this push:
new 0b9e6d9 Updated PIP 38: Batch Receiving Messages (markdown)
0b9e6d9 is described below
commit 0b9e6d91709f50a0bbb21f0b54518fa4fbbeb1b3
Author: lipenghui <[email protected]>
AuthorDate: Mon Jul 1 09:56:45 2019 +0800
Updated PIP 38: Batch Receiving Messages (markdown)
---
PIP-38:-Batch-Receiving-Messages.md | 44 ++++++++++++++++++-------------------
1 file changed, 22 insertions(+), 22 deletions(-)
diff --git a/PIP-38:-Batch-Receiving-Messages.md
b/PIP-38:-Batch-Receiving-Messages.md
index 94a030f..ab0ef88 100644
--- a/PIP-38:-Batch-Receiving-Messages.md
+++ b/PIP-38:-Batch-Receiving-Messages.md
@@ -5,7 +5,7 @@
## Motivation
-Batch processing is a commonly used to improve throughput, support batch
receving in client can be better adapted to user's existing batch
operations(batch insert data to database or other bulk APIs). At present,
pulsar client provide the ability to receive a single message. If users want to
take advantage of batch operating advantages, need to implement a message
collector him self.
+Batch processing is commonly used to improve throughput, support batch
receiving in client can be better adapted to user's existing batch
operations(batch insert data to database or other bulk APIs). At present,
pulsar client provides the ability to receive a single message. If users want
to take advantage of batch operating advantages, need to implement a message
collector himself.
For throughput optimization in the future will benefit from batch receiving ,
it can allow lazy deserialization and object creation, can also reduce
`incomingMessages` enqueue and dequeue times
@@ -17,7 +17,7 @@ Batch receiving should have the following capabilities:
- Multiple messages can be received at a time
- Users can set the max number or size of messages received in batches for
consumers
-- Provide an timeout mechanism to avoid waiting indefinitely
+- Provide a timeout mechanism to avoid waiting indefinitely
## Usage
@@ -41,11 +41,11 @@ Consumer<String> consumer =
pulsarClient.newConsumer(Schema.STRING)
.subscribe();
```
-Batch receive policy can met multiple use cases:
+Batch receive policy can meet multiple use cases:
**Fixed number of messages**
-Consumer will be blocked until has enough number of messages avaliable.
+Consumer will be blocked until has enough number of messages available.
```java
BatchReceivePolicy.builder().maxNumMessages(10).build();
@@ -53,7 +53,7 @@ BatchReceivePolicy.builder().maxNumMessages(10).build();
**Fixed bytes of messages**
-Consumer will be blocked until has enough bytes of messages avaliable.
+Consumer will be blocked until has enough bytes of messages available.
```java
BatchReceivePolicy.builder().maxNumBytes(1024 * 1024 * 10).build();
@@ -73,7 +73,7 @@ BatchReceivePolicy.builder().timeout(1,
TimeUnit.SECONDS).build();
> Note:
>
-> This way will not limit the number or bytes of a messages batch, ensure have
enough memory resources to maintain the messages batch in a single time period.
+> This way will not limit the number or bytes of a message batch, ensure have
enough memory resources to maintain the messages batch in a single time period.
**Hybrid control**
@@ -115,7 +115,7 @@ public interface Messages<T> extends Iterable<Message<T>> {
* Batch receiving messages
* <p>
* This calls blocks until has enough messages or wait timeout, more details
to see {@link BatchReceivePolicy}
- *
+
* @return messages
* @since 2.5.0
* @throws PulsarClientException
@@ -140,7 +140,7 @@ CompletableFuture<Messages<T>> batchReceiveAsync();
/**
* Acknowledge the consumption of {@link Messages}
- *
+
* @param messages messages
* @throws PulsarClientException.AlreadyClosedException
* if the consumer was already closed
@@ -155,16 +155,16 @@ void acknowledge(Messages<?> messages) throws
PulsarClientException;
* with {@link ConsumerBuilder#negativeAckRedeliveryDelay(long, TimeUnit)}.
* <p>
* This call is not blocking.
- *
+
* <p>
* Example of usage:
* <pre><code>
* while (true) {
* Messages<String> msgs = consumer.batchReceive();
- *
+
* try {
* // Process message...
- *
+
* consumer.acknowledge(msgs);
* } catch (Throwable t) {
* log.warn("Failed to process message");
@@ -172,7 +172,7 @@ void acknowledge(Messages<?> messages) throws
PulsarClientException;
* }
* }
* </code></pre>
- *
+
* @param message
* The {@code Message} to be acknowledged
*/
@@ -184,31 +184,31 @@ void negativeAcknowledge(Messages<?> messages);
```java
/**
* Configuration for message batch receive {@link Consumer#batchReceive()}
{@link Consumer#batchReceiveAsync()}.
- *
+
* Batch receive policy can limit the number and size of messages in a single
batch, and can specify a timeout
* for waiting for enough messages for this batch.
- *
+
* This batch receive will be completed as long as any one of the
* conditions(has enough number of messages, has enough of size of messages,
wait timeout) is met.
- *
+
* Examples:
- *
+
* 1.If set maxNumberOfMessages = 10, maxSizeOfMessages = 1MB and without
timeout, it
* means {@link Consumer#batchReceive()} will always wait until there is
enough messages.
- *
+
* 2.If set maxNumberOfMessages = 0, maxSizeOfMessages = 0 and timeout =
100ms, it
- * means {@link Consumer#batchReceive()} will waiting for 100ms whether or not
there is enough messages.
- *
+ * means {@link Consumer#batchReceive()} will wait for 100ms whether or not
there is enough messages.
+
* Note:
* Must specify messages limitation(maxNumberOfMessages, maxSizeOfMessages) or
wait timeout.
* Otherwise, {@link Messages} ingest {@link Message} will never end.
- *
+
* @since 2.5.0
*/
public class BatchReceivePolicy {
/**
* Default batch receive policy
- *
+
* Max number of messages: 100
* Max size of messages: 10MB
* Timeout: 100ms
@@ -245,4 +245,4 @@ Ideally the pulsar client implementation should
1. Keep a queue of `Messages`. Each `Messages` is a message batch or multiple
message batches.
2. On receiving individual message, it polls a `Messages` from the queue, and
poll a message out of the `Messages`.
-This can allow lazy deserialization and object creation, and it will increase
the throughput using batch receive api because your cpu cycles can be reduced.
\ No newline at end of file
+This can allow lazy deserialization and object creation, and it will increase
the throughput using batch receive api because your cpu cycles can be reduced.
