This is an automated email from the ASF dual-hosted git repository.
liuyu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 9fe8a80 [website][upgrade]feat: docs migration - 2.7.1 / transaction
(#12579)
9fe8a80 is described below
commit 9fe8a80cb69eb25760792cfba419989febd992a3
Author: Li Li <[email protected]>
AuthorDate: Wed Nov 3 12:28:31 2021 +0800
[website][upgrade]feat: docs migration - 2.7.1 / transaction (#12579)
* [website][upgrade]feat: docs migration - 2.7.1 / io
Signed-off-by: LiLi <[email protected]>
* [website][upgrade]feat: docs migration - 2.7.1 / sql
Signed-off-by: LiLi <[email protected]>
* [website][upgrade]feat: docs migration - 2.7.1 / tiered-storage
Signed-off-by: LiLi <[email protected]>
* [website][upgrade]feat: docs migration - 2.7.1 / transaction
Signed-off-by: LiLi <[email protected]>
---
.../version-2.7.1/concepts-transactions.md | 34 ++++
.../version-2.7.1/transaction-api.md | 178 +++++++++++++++++++++
.../version-2.7.1/transaction-guarantee.md | 21 +++
.../versioned_sidebars/version-2.7.1-sidebars.json | 18 +++
4 files changed, 251 insertions(+)
diff --git
a/site2/website-next/versioned_docs/version-2.7.1/concepts-transactions.md
b/site2/website-next/versioned_docs/version-2.7.1/concepts-transactions.md
new file mode 100644
index 0000000..9e44d4f
--- /dev/null
+++ b/site2/website-next/versioned_docs/version-2.7.1/concepts-transactions.md
@@ -0,0 +1,34 @@
+---
+id: transactions
+title: Transactions
+sidebar_label: "Overview"
+original_id: transactions
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+
+Transactional semantics enable event streaming applications to consume,
process, and produce messages in one atomic operation. In Pulsar, a producer or
consumer can work with messages across multiple topics and partitions and
ensure those messages are processed as a single unit.
+
+The following concepts help you understand Pulsar transactions.
+
+## Transaction coordinator and transaction log
+The transaction coordinator maintains the topics and subscriptions that
interact in a transaction. When a transaction is committed, the transaction
coordinator interacts with the topic owner broker to complete the transaction.
+
+The transaction coordinator maintains the entire life cycle of transactions,
and prevents a transaction from incorrect status.
+
+The transaction coordinator handles transaction timeout, and ensures that the
transaction is aborted after a transaction timeout.
+
+All the transaction metadata is persisted in the transaction log. The
transaction log is backed by a Pulsar topic. After the transaction coordinator
crashes, it can restore the transaction metadata from the transaction log.
+
+## Transaction ID
+The transaction ID (TxnID) identifies a unique transaction in Pulsar. The
transaction ID is 128-bit. The highest 16 bits are reserved for the ID of the
transaction coordinator, and the remaining bits are used for monotonically
increasing numbers in each transaction coordinator. It is easy to locate the
transaction crash with the TxnID.
+
+## Transaction buffer
+Messages produced within a transaction are stored in the transaction buffer.
The messages in transaction buffer are not materialized (visible) to consumers
until the transactions are committed. The messages in the transaction buffer
are discarded when the transactions are aborted.
+
+## Pending acknowledge state
+Message acknowledges within a transaction are maintained by the pending
acknowledge state before the transaction completes. If a message is in the
pending acknowledge state, the message cannot be acknowledged by other
transactions until the message is removed from the pending acknowledge state.
+
+The pending acknowledge state is persisted to the pending acknowledge log. The
pending acknowledge log is backed by a Pulsar topic. A new broker can restore
the state from the pending acknowledge log to ensure the acknowledgement is not
lost.
diff --git a/site2/website-next/versioned_docs/version-2.7.1/transaction-api.md
b/site2/website-next/versioned_docs/version-2.7.1/transaction-api.md
new file mode 100644
index 0000000..ba8deda
--- /dev/null
+++ b/site2/website-next/versioned_docs/version-2.7.1/transaction-api.md
@@ -0,0 +1,178 @@
+---
+id: transactions-api
+title: Transactions API (Developer Preview)
+sidebar_label: "Transactions API"
+original_id: transactions-api
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+
+All messages in a transaction is available only to consumers after the
transaction is committed. If a transaction is aborted, all the writes and
acknowledgments in this transaction rollback.
+
+Currently, Pulsar transaction is a developer preview feature. It is disabled
by default. You can enable the feature and use transactions in your application
in development environment.
+
+## Prerequisites
+1. To enable transactions in Pulsar, you need to configure the parameter in
the `broker.conf` file.
+
+```
+
+transactionCoordinatorEnabled=true
+
+```
+
+2. Initialize transaction coordinator metadata, so the transaction
coordinators can leverage advantages of the partitioned topic, such as load
balance.
+
+```
+
+bin/pulsar initialize-transaction-coordinator-metadata -cs 127.0.0.1:2181 -c
standalone
+
+```
+
+After initializing transaction coordinator metadata, you can use the
transactions API. The following APIs are available.
+
+## Initialize Pulsar client
+
+You can enable transaction for transaction client and initialize transaction
coordinator client.
+
+```
+
+PulsarClient pulsarClient = PulsarClient.builder()
+ .serviceUrl("pulsar://localhost:6650")
+ .enableTransaction(true)
+ .build();
+
+```
+
+## Start transactions
+You can start transaction in the following way.
+
+```
+
+Transaction txn = pulsarClient
+ .newTransaction()
+ .withTransactionTimeout(5, TimeUnit.MINUTES)
+ .build()
+ .get();
+
+```
+
+## Produce transaction messages
+
+A transaction parameter is required when producing new transaction messages.
The semantic of the transaction messages in Pulsar is `read-committed`, so the
consumer cannot receive the ongoing transaction messages before the transaction
is committed.
+
+```
+
+producer.newMessage(txn).value("Hello Pulsar
Transaction".getBytes()).sendAsync();
+
+```
+
+## Acknowledge the messages with the transaction
+
+The transaction acknowledgement requires a transaction parameter. The
transaction acknowledgement marks the messages state to pending-ack state. When
the transaction is committed, the pending-ack state becomes ack state. If the
transaction is aborted, the pending-ack state becomes unack state.
+
+```
+
+Message<byte[]> message = consumer.receive();
+consumer.acknowledgeAsync(message.getMessageId(), txn);
+
+```
+
+## Commit transactions
+
+When the transaction is committed, consumers receive the transaction messages
and the pending-ack state becomes ack state.
+
+```
+
+txn.commit().get();
+
+```
+
+## Abort transaction
+
+When the transaction is aborted, the transaction acknowledgement is canceled
and the pending-ack messages are redelivered.
+
+```
+
+txn.abort().get();
+
+```
+
+### Example
+The following example shows how messages are processed in transaction.
+
+```
+
+PulsarClient pulsarClient = PulsarClient.builder()
+ .serviceUrl(getPulsarServiceList().get(0).getBrokerServiceUrl())
+ .statsInterval(0, TimeUnit.SECONDS)
+ .enableTransaction(true)
+ .build();
+
+String sourceTopic = "public/default/source-topic";
+String sinkTopic = "public/default/sink-topic";
+
+Producer<String> sourceProducer = pulsarClient
+ .newProducer(Schema.STRING)
+ .topic(sourceTopic)
+ .create();
+sourceProducer.newMessage().value("hello pulsar transaction").sendAsync();
+
+Consumer<String> sourceConsumer = pulsarClient
+ .newConsumer(Schema.STRING)
+ .topic(sourceTopic)
+ .subscriptionName("test")
+ .subscriptionType(SubscriptionType.Shared)
+ .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
+ .subscribe();
+
+Producer<String> sinkProducer = pulsarClient
+ .newProducer(Schema.STRING)
+ .topic(sinkTopic)
+ .sendTimeout(0, TimeUnit.MILLISECONDS)
+ .create();
+
+Transaction txn = pulsarClient
+ .newTransaction()
+ .withTransactionTimeout(5, TimeUnit.MINUTES)
+ .build()
+ .get();
+
+// source message acknowledgement and sink message produce belong to one
transaction,
+// they are combined into an atomic operation.
+Message<String> message = sourceConsumer.receive();
+sourceConsumer.acknowledgeAsync(message.getMessageId(), txn);
+sinkProducer.newMessage(txn).value("sink data").sendAsync();
+
+txn.commit().get();
+
+```
+
+## Enable batch messages in transactions
+
+To enable batch messages in transactions, you need to enable the batch index
acknowledgement feature. The transaction acks check whether the batch index
acknowledgement conflicts.
+
+To enable batch index acknowledgement, you need to set
`acknowledgmentAtBatchIndexLevelEnabled` to `true` in the `broker.conf` or
`standalone.conf` file.
+
+```
+
+acknowledgmentAtBatchIndexLevelEnabled=true
+
+```
+
+And then you need to call the `enableBatchIndexAcknowledgment(true)` method in
the consumer builder.
+
+```
+
+Consumer<byte[]> sinkConsumer = pulsarClient
+ .newConsumer()
+ .topic(transferTopic)
+ .subscriptionName("sink-topic")
+ .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
+ .subscriptionType(SubscriptionType.Shared)
+ .enableBatchIndexAcknowledgment(true) // enable batch index
acknowledgement
+ .subscribe();
+
+```
+
diff --git
a/site2/website-next/versioned_docs/version-2.7.1/transaction-guarantee.md
b/site2/website-next/versioned_docs/version-2.7.1/transaction-guarantee.md
new file mode 100644
index 0000000..b75c94a
--- /dev/null
+++ b/site2/website-next/versioned_docs/version-2.7.1/transaction-guarantee.md
@@ -0,0 +1,21 @@
+---
+id: transactions-guarantee
+title: Transactions Guarantee
+sidebar_label: "Transactions Guarantee"
+original_id: transactions-guarantee
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+
+Pulsar transactions support the following guarantee.
+
+## Atomic multi-partition writes and multi-subscription acknowledges
+Transactions enable atomic writes to multiple topics and partitions. A batch
of messages in a transaction can be received from, produced to, and
acknowledged by many partitions. All the operations involved in a transaction
succeed or fail as a single unit.
+
+## Read transactional message
+All the messages in a transaction are available only for consumers until the
transaction is committed.
+
+## Acknowledge transactional message
+A message is acknowledged successfully only once by a consumer under the
subscription when acknowledging the message with the transaction ID.
\ No newline at end of file
diff --git a/site2/website-next/versioned_sidebars/version-2.7.1-sidebars.json
b/site2/website-next/versioned_sidebars/version-2.7.1-sidebars.json
index aa9f4c8..2ac7003 100644
--- a/site2/website-next/versioned_sidebars/version-2.7.1-sidebars.json
+++ b/site2/website-next/versioned_sidebars/version-2.7.1-sidebars.json
@@ -213,6 +213,24 @@
"id": "version-2.7.1/tiered-storage-azure"
}
]
+ },
+ {
+ "type": "category",
+ "label": "Transactions",
+ "items": [
+ {
+ "type": "doc",
+ "id": "version-2.7.1/transactions"
+ },
+ {
+ "type": "doc",
+ "id": "version-2.7.1/transactions-guarantee"
+ },
+ {
+ "type": "doc",
+ "id": "version-2.7.1/transactions-api"
+ }
+ ]
}
]
}
\ No newline at end of file
