This is an automated email from the ASF dual-hosted git repository.
liuyu pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new f0af616eeba [improve][doc] SEO for Tiered Storage (#684)
f0af616eeba is described below
commit f0af616eeba17a38d8329d1d1bd5dee48a451b7a
Author: Zhang Yuxuan <[email protected]>
AuthorDate: Mon Sep 4 10:40:14 2023 +0800
[improve][doc] SEO for Tiered Storage (#684)
---
docs/tiered-storage-aliyun.md | 5 +++--
docs/tiered-storage-aws.md | 5 +++--
docs/tiered-storage-azure.md | 5 +++--
docs/tiered-storage-gcs.md | 5 +++--
docs/tiered-storage-overview.md | 29 ++++++++++++++++++++++-------
docs/tiered-storage-s3.md | 5 +++--
6 files changed, 37 insertions(+), 17 deletions(-)
diff --git a/docs/tiered-storage-aliyun.md b/docs/tiered-storage-aliyun.md
index 9fa839028c7..b0639b67e48 100644
--- a/docs/tiered-storage-aliyun.md
+++ b/docs/tiered-storage-aliyun.md
@@ -2,20 +2,21 @@
id: tiered-storage-aliyun
title: Use Aliyun OSS offloader with Pulsar
sidebar_label: "Aliyun OSS offloader"
+description: Learn to install and configure Aliyun OSS offloader with Pulsar.
---
This chapter guides you through every step of installing and configuring the
Aliyun Object Storage Service (OSS) offloader and using it with Pulsar.
## Installation
-Follow the steps below to install the Aliyun OSS offloader.
-
### Prerequisite
- Pulsar: 2.8.0 or later versions
### Steps
+Follow the steps below to install the Aliyun OSS offloader.
+
1. [Download the Pulsar
tarball](getting-started-standalone.md#download-pulsar-distribution).
2. Download and untar the Pulsar offloaders package, then copy the Pulsar
offloaders as `offloaders` in the Pulsar directory. See [Install tiered storage
offloaders](tiered-storage-overview.md#how-to-install-tiered-storage-offloaders).
diff --git a/docs/tiered-storage-aws.md b/docs/tiered-storage-aws.md
index ce802e0e8d9..9527811a26e 100644
--- a/docs/tiered-storage-aws.md
+++ b/docs/tiered-storage-aws.md
@@ -2,20 +2,21 @@
id: tiered-storage-aws
title: Use AWS S3 offloader with Pulsar
sidebar_label: "AWS S3 offloader"
+description: Learn to install and configure AWS S3 offloader with Pulsar.
---
This chapter guides you through every step of installing and configuring the
AWS S3 offloader and using it with Pulsar.
## Installation
-Follow the steps below to install the AWS S3 offloader.
-
### Prerequisite
- Pulsar: 2.4.2 or later versions
### Steps
+Follow the steps below to install the AWS S3 offloader.
+
1. [Download the Pulsar
tarball](getting-started-standalone.md#download-pulsar-distribution).
2. Download and untar the Pulsar offloaders package, then copy the Pulsar
offloaders as `offloaders` in the Pulsar directory. See [Install tiered storage
offloaders](tiered-storage-overview.md#how-to-install-tiered-storage-offloaders).
diff --git a/docs/tiered-storage-azure.md b/docs/tiered-storage-azure.md
index 4c7b2e55167..d837cce7d82 100644
--- a/docs/tiered-storage-azure.md
+++ b/docs/tiered-storage-azure.md
@@ -2,20 +2,21 @@
id: tiered-storage-azure
title: Use Azure BlobStore offloader with Pulsar
sidebar_label: "Azure BlobStore offloader"
+description: Learn to install and configure Azure BlobStore offloader with
Pulsar.
---
This chapter guides you through every step of installing and configuring the
Azure BlobStore offloader and using it with Pulsar.
## Installation
-Follow the steps below to install the Azure BlobStore offloader.
-
### Prerequisite
- Pulsar: 2.6.2 or later versions
### Steps
+Follow the steps below to install the Azure BlobStore offloader.
+
1. [Download the Pulsar
tarball](getting-started-standalone.md#download-pulsar-distribution).
2. Download and untar the Pulsar offloaders package, then copy the Pulsar
offloaders as `offloaders` in the Pulsar directory. See [Install tiered storage
offloaders](tiered-storage-overview.md#how-to-install-tiered-storage-offloaders).
diff --git a/docs/tiered-storage-gcs.md b/docs/tiered-storage-gcs.md
index 9395a792cc4..20ca5a03215 100644
--- a/docs/tiered-storage-gcs.md
+++ b/docs/tiered-storage-gcs.md
@@ -2,20 +2,21 @@
id: tiered-storage-gcs
title: Use GCS offloader with Pulsar
sidebar_label: "GCS offloader"
+description: Learn to install and configure GCS offloader with Pulsar.
---
This chapter guides you through every step of installing and configuring the
GCS offloader and using it with Pulsar.
## Installation
-Follow the steps below to install the GCS offloader.
-
### Prerequisite
- Pulsar: 2.4.2 or later versions
### Steps
+Follow the steps below to install the GCS offloader.
+
1. [Download the Pulsar
tarball](getting-started-standalone.md#download-pulsar-distribution).
2. Download and untar the Pulsar offloaders package, then copy the Pulsar
offloaders as `offloaders` in the Pulsar directory. See [Install tiered storage
offloaders](tiered-storage-overview.md#how-to-install-tiered-storage-offloaders).
diff --git a/docs/tiered-storage-overview.md b/docs/tiered-storage-overview.md
index d0afd3689d3..9bd8afaec02 100644
--- a/docs/tiered-storage-overview.md
+++ b/docs/tiered-storage-overview.md
@@ -2,6 +2,7 @@
id: tiered-storage-overview
title: Overview of tiered storage
sidebar_label: "Overview"
+description: Get a comprehensive understanding of use cases, installation
methods, and working principles of Pulsar tiered storage.
---
Pulsar's **Tiered Storage** feature allows older backlog data to be moved from
BookKeeper to long-term and cheaper storage, while still allowing clients to
access the backlog as if nothing has changed.
@@ -30,13 +31,15 @@ For example, if you have a topic containing user actions
that you use to train y
## How to install tiered storage offloaders?
-Pulsar releases a separate binary distribution, containing the tiered storage
offloaders. To enable those offloaders, you need to download the offloaders
tarball release:
+Pulsar releases a separate binary distribution, containing the tiered storage
offloaders. To enable those offloaders, you need to complete the following
steps.
+
+1. Download the offloaders tarball release.
```bash
wget pulsar:offloader_release_url
```
-After you download the tarball, untar the offloaders package and copy the
offloaders as `offloaders` in the pulsar directory:
+2. Untar the offloaders package and copy the offloaders as `offloaders` in the
pulsar directory.
```bash
tar xvfz apache-pulsar-offloaders-@pulsar:[email protected]
@@ -60,14 +63,26 @@ For more information on how to configure tiered storage,
see [Tiered storage coo
A topic in Pulsar is backed by a **log**, known as a **managed ledger**. This
log is composed of an ordered list of segments. Pulsar only writes to the final
segment of the log. All previous segments are sealed. The data within the
segment is immutable. This is known as a **segment-oriented architecture**.
-
+
+
+Tiered storage works as follows:
+
+1. The tiered storage offloading mechanism takes advantage of the
segment-oriented architecture.
+
+ When offloading is requested, the segments of the log are copied one by one
to tiered storage. All segments of the log (apart from the current segment)
written to tiered storage can be offloaded.
+
+2. Data written to BookKeeper is replicated to 3 physical machines by default.
+
+ However, once a segment is sealed in BookKeeper, it becomes immutable and
can be copied to long-term storage. Long-term storage has the potential to
achieve significant cost savings.
+
+3. Before offloading ledgers to long-term storage, you need to configure
buckets, credentials, and other properties for the cloud storage service.
-The tiered storage offloading mechanism takes advantage of the
segment-oriented architecture. When offloading is requested, the segments of
the log are copied one by one to tiered storage. All segments of the log (apart
from the current segment) written to tiered storage can be offloaded.
+4. Additionally, Pulsar uses multi-part objects to upload the segment data and
brokers may crash while uploading the data.
-Data written to BookKeeper is replicated to 3 physical machines by default.
However, once a segment is sealed in BookKeeper, it becomes immutable and can
be copied to long-term storage. Long-term storage has the potential to achieve
significant cost savings.
+ It is recommended that you add a life cycle rule for your bucket to expire
incomplete multi-part upload after a day or two days to avoid getting charged
for incomplete uploads.
-Before offloading ledgers to long-term storage, you need to configure buckets,
credentials, and other properties for the cloud storage service. Additionally,
Pulsar uses multi-part objects to upload the segment data and brokers may crash
while uploading the data. It is recommended that you add a life cycle rule for
your bucket to expire incomplete multi-part upload after a day or two days to
avoid getting charged for incomplete uploads. Moreover, you can trigger the
offloading operation [...]
+5. Moreover, you can trigger the offloading operation manually (via REST API
or CLI) or automatically (via CLI).
-After offloading ledgers to long-term storage, you can still query data in the
offloaded ledgers with Pulsar SQL.
+6. After offloading ledgers to long-term storage, you can still query data in
the offloaded ledgers with Pulsar SQL.
For more information about tiered storage for Pulsar topics, see
[PIP-17](https://github.com/apache/pulsar/wiki/PIP-17:-Tiered-storage-for-Pulsar-topics)
and [offload metrics](reference-metrics.md#offload-metrics).
\ No newline at end of file
diff --git a/docs/tiered-storage-s3.md b/docs/tiered-storage-s3.md
index 198813cc1c4..cc47cf57011 100644
--- a/docs/tiered-storage-s3.md
+++ b/docs/tiered-storage-s3.md
@@ -2,6 +2,7 @@
id: tiered-storage-s3
title: Use S3 offloader with Pulsar
sidebar_label: "S3 offloader"
+description: Learn to install and configure S3 offloader with Pulsar.
---
S3 offloader is introduced to serve S3-compatible storage, which means that
the storage employs the S3 API as its "language" and applications that speak
the S3 API are able to plug and play with S3-compatible storage.
@@ -10,14 +11,14 @@ This chapter guides you through every step of installing
and configuring the S3
## Installation
-Follow the steps below to install the S3 offloader.
-
### Prerequisite
- Pulsar: 2.9.3 or later versions
### Steps
+Follow the steps below to install the S3 offloader.
+
1. [Download the Pulsar
tarball](getting-started-standalone.md#download-pulsar-distribution).
2. Download and untar the Pulsar offloaders package, then copy the Pulsar
offloaders as `offloaders` in the Pulsar directory. See [Install tiered storage
offloaders](tiered-storage-overview.md#how-to-install-tiered-storage-offloaders).
