jiangphcn closed pull request #285: Couchdb 3326 clustered purge
URL: https://github.com/apache/couchdb-documentation/pull/285
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/images/purge-checkpoint-docs.png b/images/purge-checkpoint-docs.png
new file mode 100644
index 0000000..943db23
Binary files /dev/null and b/images/purge-checkpoint-docs.png differ
diff --git a/images/rev-tree1.png b/images/rev-tree1.png
new file mode 100644
index 0000000..467f69e
Binary files /dev/null and b/images/rev-tree1.png differ
diff --git a/images/rev-tree2.png b/images/rev-tree2.png
new file mode 100644
index 0000000..e77ca3b
Binary files /dev/null and b/images/rev-tree2.png differ
diff --git a/images/rev-tree3.png b/images/rev-tree3.png
new file mode 100644
index 0000000..fa97c7d
Binary files /dev/null and b/images/rev-tree3.png differ
diff --git a/src/api/database/misc.rst b/src/api/database/misc.rst
index 2337a16..3cbfed7 100644
--- a/src/api/database/misc.rst
+++ b/src/api/database/misc.rst
@@ -17,40 +17,23 @@
==============
.. http:post:: /{db}/_purge
- :synopsis: Purges some historical documents entirely from database history
+ :synopsis: Purges documents entirely from database
- A database purge permanently removes the references to deleted documents
- from the database. Normal deletion of a document within CouchDB does not
+ A database purge permanently removes the references to documents
+ in the database. Normal deletion of a document within CouchDB does not
remove the document from the database, instead, the document is marked as
``_deleted=true`` (and a new revision is created). This is to ensure that
deleted documents can be replicated to other databases as having been
deleted. This also means that you can check the status of a document and
identify that the document has been deleted by its absence.
- .. warning::
- Purging a document from a database should only be done as a last resort
- when sensitive information has been introduced inadvertently into a
- database. In clustered or replicated environments it is very difficult
- to guarantee that a particular purged document has been removed from
- all replicas. Do not rely on this API as a way of doing secure
- deletion.
+ The purge request must include the document IDs, and for each
+ document ID, one or more revisions that must be purged. Documents can be
+ previously deleted, but it is not necessary. Revisions must be leaf
+ revisions.
- The purge operation removes the references to the deleted documents from
- the database. The purging of old documents is not replicated to other
- databases. If you are replicating between databases and have deleted a
- large number of documents you should run purge on each database.
-
- .. note::
- Purging documents does not remove the space used by them on disk. To
- reclaim disk space, you should run a database compact (see
- :ref:`api/db/compact`), and compact views
- (see :ref:`api/db/compact/ddoc`).
-
- The format of the request must include the document ID and one or more
- revisions that must be purged.
-
- The response will contain the purge sequence number, and a list of the
- document IDs and revisions successfully purged.
+ The response will contain a list of the document IDs and revisions
+ successfully purged.
:param db: Database name
:<header Accept: - :mimetype:`application/json`
@@ -59,11 +42,14 @@
:<json object: Mapping of document ID to list of revisions to purge
:>header Content-Type: - :mimetype:`application/json`
- :mimetype:`text/plain; charset=utf-8`
- :>json number purge_seq: Purge sequence number
+ :>json string purge_seq: Purge sequence string
:>json object purged: Mapping of document ID to list of purged revisions
- :code 200: Request completed successfully
+ :code 201: Request completed successfully
+ :code 202: Request was accepted, and was completed successfully on at least
+ one replica, but quorum was not reached.
:code 400: Invalid database name or JSON payload
:code 415: Bad :header:`Content-Type` value
+ :code 500: Internal server error or timeout
**Request**:
@@ -78,7 +64,7 @@
{
"c6114c65e295552ab1019e2b046b10e": [
"3-b06fcd1c1c9e0ec7c480ee8aa467bf3b",
- "3-0e871ef78849b0c206091f1a7af6ec41"
+ "3-c50a32451890a3f1c3e423334cc92745"
]
}
@@ -86,22 +72,85 @@
.. code-block:: http
- HTTP/1.1 200 OK
+ HTTP/1.1 201 Created
Cache-Control: must-revalidate
- Content-Length: 103
+ Content-Length: 107
Content-Type: application/json
- Date: Mon, 12 Aug 2013 10:53:24 GMT
- Server: CouchDB (Erlang/OTP)
+ Date: Fri, 02 Jun 2017 18:55:54 GMT
+ Server: CouchDB/2.0.0-2ccd4bf (Erlang OTP/18)
{
- "purge_seq":3,
- "purged":{
- "c6114c65e295552ab1019e2b046b10e": [
- "3-b06fcd1c1c9e0ec7c480ee8aa467bf3b"
- ]
+ "purge_seq": "3",
+ "purged": {
+ "c6114c65e295552ab1019e2b046b10e": {
+ "purged": [
+ "3-c50a32451890a3f1c3e423334cc92745"
+ ],
+ "ok": true
}
+ }
}
+.. figure:: ../../../images/rev-tree1.png
+ :align: center
+ :alt: Document Revision Tree 1
+
+ Document Revision Tree 1
+
+For example, given the above purge tree and issuing the above purge request,
+the whole document will be purged, as it contains only a single branch with a
+leaf revision `3-c50a32451890a3f1c3e423334cc92745` that will be purged.
+As a result of this purge operation, a document with
+`_id:c6114c65e295552ab1019e2b046b10e` will be completely removed from the
+database's document b+tree, and sequence b+tree. It will not be available
+through `_all_docs` or `_changes` endpoints, as though this document never
+existed. Also as a result of purge operation, the database's `purge_seq` and
+`update_seq` will be increased.
+
+Notice, how revision `3-b06fcd1c1c9e0ec7c480ee8aa467bf3b` was ignored.
Revisions
+that have already been purged and non-leaf revisions are ignored in a purge
+request.
+
+If a document has two conflict revisions with the following revision history:
+
+.. figure:: ../../../images/rev-tree2.png
+ :align: center
+ :alt: Document Revision Tree 1
+
+ Document Revision Tree 2
+
+the above purge request will purge only one branch, leaving the document's
+revision tree with only a single branch:
+
+.. figure:: ../../../images/rev-tree3.png
+ :align: center
+ :alt: Document Revision Tree 3
+
+ Document Revision Tree 3
+
+As a result of this purge operation, a new updated version of the document will
+be available in `_all_docs` and `_changes`, creating a new record in
`_changes`.
+The database's `purge_seq` and `update_seq` will be increased.
+
+Internal Replication
+======================
+Purges are automatically replicated between replicas of the same database. Each
+database has an internal purge tree that stores a certain number of the most
+recent purges. This allows internal synchonization between replicas of the same
+database.
+
+External Replication
+======================
+Purge operations are not replicated to other external databases. External
+replication works by identifying a source's document revisions that are missing
+on target, and copying these revisions from source to target. A purge operation
+completely purges revisions from a document's purge tree making external
+replication of purges impossible.
+
+ .. note::
+ If you need a purge to be effective across multiple effective databases,
you
+ must run the purge separately on each of the databases.
+
Updating Indexes
================
@@ -109,16 +158,122 @@ The number of purges on a database is tracked using a
purge sequence. This is
used by the view indexer to optimize the updating of views that contain the
purged documents.
-When the indexer identifies that the purge sequence on a database has changed,
-it compares the purge sequence of the database with that stored in the view
-index. If the difference between the stored sequence and database is sequence
-is only 1, then the indexer uses a cached list of the most recently purged
-documents, and then removes these documents from the index individually. This
-prevents completely rebuilding the index from scratch.
+Each internal database indexer, including the view indexer, keeps its own purge
+sequence. The purge sequence stored in the index can be much smaller than the
+database's purge sequence up to the number of purge requests allowed to be
+stored in the purge trees of the database. Multiple purge requests can be
+processed by the indexer without incurring a rebuild of the index. The index
+will be updated according to these purge requests.
+
+The index of documents is based on the winner of the revision tree. Depending
on
+which revision is specified in the purge request, the index update observes the
+following behavior:
+
+- If the winner of the revision tree is not specified in the purge request,
+ there is no change to the index record of this document.
+- If the winner of the revision tree is specified in the purge request, and
+ there is still a revision left after purging, the index record of the
document
+ will be built according to the new winner of the revision tree.
+- If all revisions of the document are specified in the purge request, the
index
+ record of the document will be deleted. The document will no longer be found
+ in searches.
+
+.. _api/db/_purged_infos_limit:
+
+==============================
+``/db/_purged_infos_limit``
+==============================
+
+.. http:get:: /{db}/_purged_infos_limit
+ :synopsis: Returns the limit of historical purges to store in the database
+
+ Gets the current ``purged_infos_limit`` (purged documents limit) setting,
+ the maximum number of historical purges (purged document Ids with their
+ revisions) that can be stored in the database.
+
+ :param db: Database name
+ :<header Accept: - :mimetype:`application/json`
+ - :mimetype:`text/plain`
+ :>header Content-Type: - :mimetype:`application/json`
+ - :mimetype:`text/plain; charset=utf-8`
+ :code 200: Request completed successfully
+
+ **Request**:
+
+ .. code-block:: http
+
+ GET /db/_purged_infos_limit HTTP/1.1
+ Accept: application/json
+ Host: localhost:5984
-If the difference between the stored sequence number and current database
-sequence is greater than 1, then the view index is entirely rebuilt. This is
-an expensive operation as every document in the database must be examined.
+ **Response**:
+
+ .. code-block:: http
+
+ HTTP/1.1 200 OK
+ Cache-Control: must-revalidate
+ Content-Length: 5
+ Content-Type: application/json
+ Date: Wed, 14 Jun 2017 14:43:42 GMT
+ Server: CouchDB (Erlang/OTP)
+
+ 1000
+
+.. http:put:: /{db}/_purged_infos_limit
+ :synopsis: Sets the limit of historical purges to store in the database
+
+ Sets the maximum number of purges (requested purged Ids with their
+ revisions) that will be tracked in the database, even after compaction has
+ occurred. You can set the purged documents limit on a database with a
scalar
+ integer of the limit that you want to set as the request body.
+
+ The default value of historical stored purges is 1000. This means up to
1000
+ purges can be synchronized between replicas of the same databases in case
of
+ one of the replicas was down when purges occurred.
+
+ This request sets the soft limit for stored purges. During the compaction
+ CouchDB will try to keep only `_purged_infos_limit` of purges in the
+ database, but occasionally the number of stored purges can exceed this
+ value. If a database has not completed purge synchronization with active
+ indexes or active internal replications, it may temporarily store a higher
+ number of historical purges.
+
+ :param db: Database name
+ :<header Accept: - :mimetype:`application/json`
+ - :mimetype:`text/plain`
+ :<header Content-Type: :mimetype:`application/json`
+ :>header Content-Type: - :mimetype:`application/json`
+ - :mimetype:`text/plain; charset=utf-8`
+ :>json boolean ok: Operation status
+ :code 200: Request completed successfully
+ :code 400: Invalid JSON data
+
+ **Request**:
+
+ .. code-block:: http
+
+ PUT /db/_purged_infos_limit HTTP/1.1
+ Accept: application/json
+ Content-Length: 4
+ Content-Type: application/json
+ Host: localhost:5984
+
+ 1500
+
+ **Response**:
+
+ .. code-block:: http
+
+ HTTP/1.1 200 OK
+ Cache-Control: must-revalidate
+ Content-Length: 12
+ Content-Type: application/json
+ Date: Wed, 14 Jun 2017 14:45:34 GMT
+ Server: CouchDB (Erlang/OTP)
+
+ {
+ "ok": true
+ }
.. _api/db/missing_revs:
diff --git a/src/cluster/index.rst b/src/cluster/index.rst
index 4ae88ea..a66c654 100644
--- a/src/cluster/index.rst
+++ b/src/cluster/index.rst
@@ -31,3 +31,4 @@ cluster.
nodes
databases
sharding
+ purging
diff --git a/src/cluster/purging.rst b/src/cluster/purging.rst
new file mode 100644
index 0000000..996ef86
--- /dev/null
+++ b/src/cluster/purging.rst
@@ -0,0 +1,173 @@
+.. Licensed under the Apache License, Version 2.0 (the "License"); you may not
+.. use this file except in compliance with the License. You may obtain a copy
of
+.. the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+.. License for the specific language governing permissions and limitations
under
+.. the License.
+
+.. _cluster/purging:
+
+===============
+Clustered Purge
+===============
+The primary purpose of clustered purge is to clean databases that have multiple
+deleted tombstones or single documents that contain large numbers of conflicts.
+But it can also be used to purge any document (deleted or non-deleted) with any
+number of revisions.
+
+Clustered purge is designed to maintain eventual consistency and prevent
+unnecessary invalidation of secondary indexes. For this, every database keeps
+track of a certain number of historical purges requested in the database, as
+well as its current ``purge_seq``. Internal replications and secondary indexes
+process database's purges and periodically update their corresponding purge
+checkpoint documents to report ``purge_seq`` processed by them. To ensure
+eventual consistency, the database will remove stored historical purge requests
+only after they have been processed by internal replication jobs and secondary
+indexes.
+
+Internal Structures
+====================================
+To enable internal replication of purge information between nodes and secondary
+indexes, two internal purge trees were added to a database file to track
+historical purges.
+
+.. code-block:: text
+
+ purge_tree: UUID -> {PurgeSeq, DocId, Revs}
+ purge_seq_tree: PurgeSeq -> {UUID, DocId, Revs}
+
+Each interactive request to ``_purge API``, creates an ordered set of pairs on
+increasing ``purge_seq`` and purge_request, where purge_request is a tuple that
+contains docid and list of revisions. For each purge_request uuid is generated.
+A purge request is added to internal purge trees:
+a tuple ``{UUID -> {PurgeSeq, DocId, Revs}}`` is added to ``purge_tree``,
+a tuple is ``{PurgeSeq -> {UUID, DocId, Revs}}`` added ``to purge_seq_tree``.
+
+Compaction of Purges
+====================================
+
+During the compaction of the database the oldest purge requests are to be
+removed to store only ``purged_infos_limit`` number of purges in the database.
+But in order to keep the database consistent with indexes and other replicas,
+we can only remove purge requests that have already been processed by indexes
+and internal replications jobs. Thus, occasionally purge trees may store
+more than ``purged_infos_limit`` purges. If the number of stored purges in the
+database exceeds ``purged_infos_limit`` by a certain threshold, a warning is
+produced in logs signaling a problem of synchronization of database's purges
+with indexes and other replicas.
+
+Local Purge Checkpoint Documents
+====================================
+Indexes and internal replications of the database with purges create and
+periodically update local checkpoint purge documents:
+``_local/purge-$type-$hash``. These documents report the last ``purge_seq``
+processed by them and the timestamp of the last processing. An example of a
+local checkpoint purge document:
+
+.. code-block:: json
+
+ {
+ "_id": "_local/purge-mrview-86cacdfbaf6968d4ebbc324dd3723fe7",
+ "type": "mrview",
+ "purge_seq": "10",
+ "updated_on": "2018-05-09T08:41:37.183026Z",
+ "ddoc_id": "_design/foo",
+ "signature": "5d10247925f826ae3e00966ec24b7bf6"
+ }
+
+The below image shows possible local checkpoint documents that a database may
+have.
+
+.. figure:: ../../images/purge-checkpoint-docs.png
+ :align: center
+ :alt: Local Purge Checkpoint Documents
+
+ Local Purge Checkpoint Documents
+
+Internal Replication
+====================================
+Purge requests are replayed across all nodes in an eventually consistent
manner.
+Internal replication of purges consists of two steps:
+1. Pull replication. Internal replication first starts by pulling purges from
+target and applying them on source to make sure we don't reintroduce to target
+source's docs/revs that have been already purged on target. In this step, we
use
+purge checkpoint documents stored on target to keep track of the last target's
+``purge_seq`` processed by the source. We find purge requests occurred after
+this ``purge_seq``, and replay them on source. This step is done by updating
+the target's checkpoint purge documents with the latest process ``purge_seq``
+and timestamp.
+2. Push replication. Then internal replication proceeds as usual with an extra
+step inserted to push source's purge requests to target. In this step, we use
+local internal replication checkpoint documents, that are updated both on
target
+and source.
+
+Under normal conditions, an interactive purge request is already sent to every
+node containing a database shard's replica, and applied on every replica.
+Internal replication of purges between nodes is just an extra step to ensure
+consistency between replicas, where all purge requests on one node are replayed
+on another node. In order not to replay the same purge request on a replica,
+each interactive purge request is tagged with a unique ``uuid``. Internal
+replication filters out purge requests with UUIDs that already exist in the
+replica's ``purge_tree``, and applies only purge requests with UUIDs that don't
+exist in the ``purge_tree``. This is the reason why we needed to have two
+internal purge trees: 1) ``purge_tree``: ``{UUID -> {PurgeSeq, DocId, Revs}}``
+allows to quickly find purge requests with UUIDs that already exist in the
+replica; 2) ``purge_seq_tree``: ``{PurgeSeq -> {UUID, DocId, Revs}}`` allows to
+iterate from a given ``purge_seq`` to collect all purge requests happened after
+this ``purge_seq``.
+
+Indexes
+====================================
+Each purge request will bump up ``update_seq`` of the database, so that each
+secondary index is also updated in order to apply the purge requests to
maintain
+consistency within the main database.
+
+Config Settings
+====================================
+These settings ca be updated in the default.ini or local.ini:
+
+.. code-block:: text
+
+ [purge]
+ max_document_id_number = 100
+ max_revisions_number = 1000
+ allowed_purge_seq_lag = 100
+ index_lag_warn_seconds = 86400
+
+ 1. ``max_document_id_number`` (default 100)
+ Allowed maximum number of documents in one purge request
+ 2. ``max_revisions_number`` (default 1000)
+ Allowed maximum number of accumulated revisions in one purge request.
+ 3. ``index_lag_warn_seconds`` (default 86400 sec or 1 day)
+ During a database compaction, we check all checkpoint purge docs. A
client (an
+ index or internal replication job) is allowed to have the last reported
+ ``purge_seq`` to be smaller than the current database shard's
``purge_seq`` by
+ the value of ``(purged_infos_limit + index_lag_warn_seconds)``. If
the client's
+ ``purge_seq`` is even smaller, and the client has not checkpointed
within
+ ``index_lag_warn_seconds``, it prevents compaction of purge trees and
we have to
+ issue the following log warning for this client:
+
+ ::
+ Purge checkpoint '<<"_design/bar">>' not updated in 86400
seconds
+
+ If this type of log warnings occurs, check the client to see why the
processing
+ of purge requests is stalled in it.
+
+ There is a mapping relationship between design document of indexes and
local
+ checkpoint docs. If a design document of indexes is updated or
deleted, the
+ corresponding local checkpoint document should be also automatically
deleted.
+ But in an unexpected case, when a design doc was updated/deleted, but
its
+ checkpoint document still exist in a database, the following warning
will be
+ issued:
+
+ ::
+ "Invalid purge doc '<<"_design/bar">>' on database
+ <<"shards/00000000-1fffffff/testdb12.1491979089">>
+ with purge_seq '50'"
+
+ If this type of log warnings occur, remove the local purge doc from a
database.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
