Thanks Will for your comments and question. I made study on _active_tasks
endpoint in the CouchDB 4.x. Currently, we can get the information about
“indexer” and “replication” tasks in 4.x. In the response, the node information
[1] can be found like below. This gives us chance to scope _active_tasks per
node.
```
[
{
"node": "node1@127.0.0.1",
"pid": "<0.622.0>",
"changes_done": 199,
"current_version_stamp": "8131141649532-0-198",
"database": "testdb",
"db_version_stamp": "8131141649532-0-999",
"design_document": "_design/example",
"started_on": 1594703583,
"type": "indexer",
"updated_on": 1594703586
}
]
```
Best regards,
Peng Hui
[1]:
https://github.com/apache/couchdb/pull/3141/files#diff-9373f18d702a6cde8a533472cd88ca0dR290
<https://github.com/apache/couchdb/pull/3141/files#diff-9373f18d702a6cde8a533472cd88ca0dR290>
> On Sep 23, 2020, at 8:49 PM, Will Holley <willhol...@gmail.com> wrote:
>
> Thanks Peng Hui, Garren. I can definitely see value in having metrics
> exposed in Prometheus format.
>
> One aspect I'm unclear about is what _active_tasks metrics would represent.
> My expectation is that /_metrics would be scoped to a node (and then leave
> the aggregation across nodes to Prometheus or similar), whereas
> _active_tasks is scoped to a cluster. What are your thoughts?
>
> On Wed, 23 Sep 2020 at 10:18, jiangph <jiangpeng...@hotmail.com> wrote:
>
>> Thanks Joan for your quick response and suggestion. As we can see, JSON
>> is not the standard Prometheus format for _metrics endpoint. In order to be
>> compatible with many monitoring tools, what about going with Prometheus
>> standard format first and we may add JSON format support later?
>>
>> Best regards,
>> Peng Hui
>>
>>> On Sep 23, 2020, at 9:41 AM, Joan Touzet <woh...@apache.org> wrote:
>>>
>>> I like this, but not at the expense of JSON output. It would be the only
>> new API surface for CouchDB that isn't JSON-based, and there needs to be
>> excellent justification for such. Prometheus is well-known enough to be
>> supported, but we should continue to put out JSON stats for the foreseeable
>> future.
>>>
>>> I know that Prometheus can't send a header, but sending an accepts
>> application/json to /_metrics and having it send back the same data as
>> Prometheus, but in JSON, would be lovely. If you feel up to it :)
>>>
>>> -Joan "on vacation" Touzet
>>>
>>> On 2020-09-22 8:55 a.m., jiangph wrote:
>>>> Hey all,
>>>> We would like to add a Prometheus metrics endpoint for CouchDB and
>> wanted to see if the community would be interested in us contributing this
>> to CouchDB 4.x.
>>>> Prometheus is a CNCF open-source project and the Prometheus metrics
>> endpoint format is supported by many monitoring tools. Its data model is
>> based around having a metric name which then contains a label name and a
>> label value:
>>>> <metric name>{<label name>=<label value>, ...}
>>>> And it supports the Counter, Gauge, Histogram, and Summary metric types.
>>>> The idea for the new Prometheus endpoint, /_metrics, would be that the
>> endpoint is a consolidation of the _stats [1], _system [2], and
>> _active_tasks [3] endpoints.
>>>> For _stats and _system, the conversion from JSON to Prometheus-based
>> format seems to be straightforward.
>>>> JSON format:
>>>> {
>>>> "value": {
>>>> "min": 0,
>>>> "max": 0,
>>>> "arithmetic_mean": 0,
>>>> "geometric_mean": 0,
>>>> "harmonic_mean": 0,
>>>> "median": 0,
>>>> "variance": 0,
>>>> "standard_deviation": 0,
>>>> ...
>>>> "percentile": [
>>>> [
>>>> 50,
>>>> 0
>>>> ],
>>>> [
>>>> 75,
>>>> 0
>>>> ],
>>>> [
>>>> 90,
>>>> 0
>>>> ],
>>>> [
>>>> 95,
>>>> 0
>>>> ],
>>>> [
>>>> 99,
>>>> 0
>>>> ],
>>>> [
>>>> 999,
>>>> 0
>>>> ]
>>>> ],
>>>> "histogram": [
>>>> [
>>>> 0,
>>>> 0
>>>> ]
>>>> ],
>>>> }
>>>> Prometheus-based format:
>>>> couchdb_stats{value="min"} 0
>>>> couchdb_stats{value="max"} 0
>>>> couchdb_stats{value="percentile50"} 0
>>>> couchdb_stats{value="percentile75"} 0
>>>> couchdb_stats{value="percentile95"} 0
>>>> For _active_tasks, the change will be a bit more complicated, and some
>> fields will be added to labels and tags.
>>>> JSON format:
>>>> {
>>>> "checkpointed_source_seq": 68585,
>>>> "continuous": false,
>>>> "doc_id": null,
>>>> "doc_write_failures": 0,
>>>> "docs_read": 4524,
>>>> "docs_written": 4524,
>>>> "missing_revisions_found": 4524,
>>>> "pid": "<0.1538.5>",
>>>> "progress": 44,
>>>> "replication_id": "9bc1727d74d49d9e157e260bb8bbd1d5",
>>>> "revisions_checked": 4524,
>>>> "source": "mailbox",
>>>> "source_seq": 154419,
>>>> "started_on": 1376116644,
>>>> "target": "http://mailsrv:5984/mailbox <
>> http://mailsrv:5984/mailbox> <http://mailsrv:5984/mailbox <
>> http://mailsrv:5984/mailbox>>",
>>>> "type": "replication",
>>>> "updated_on": 1376116651
>>>> }
>>>> Prometheus-based would look something like:
>>>> format:couchdb_active_task{type="replication", source="mailbox",
>> target="http://mailsrv:5984/mailbox <http://mailsrv:5984/mailbox> <
>> http://mailsrv:5984/mailbox <http://mailsrv:5984/mailbox>>", docs_count =
>> "docs_read"} 4524
>>>> couchdb_active_task{type="replication", source="mailbox", target="
>> http://mailsrv:5984/mailbox <http://mailsrv:5984/mailbox> <
>> http://mailsrv:5984/mailbox <http://mailsrv:5984/mailbox>>", docs_count =
>> "docs_written"} 4524
>>>> couchdb_active_task{type="replication", source="mailbox", target="
>> http://mailsrv:5984/mailbox <http://mailsrv:5984/mailbox> <
>> http://mailsrv:5984/mailbox <http://mailsrv:5984/mailbox>>", docs_count =
>> "missing_revisions_found"} 4524
>>>> Best regards,
>>>> Garren Smith
>>>> Peng Hui Jiang
>>>> [1]
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-stats
>> <
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-stats>
>> <
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-stats
>> <
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-stats
>>>>
>>>> [2]
>> https://docs.couchdb.org/en/latest/api/server/common.html#active-tasks <
>> https://docs.couchdb.org/en/latest/api/server/common.html#active-tasks> <
>> https://docs.couchdb.org/en/latest/api/server/common.html#active-tasks <
>> https://docs.couchdb.org/en/latest/api/server/common.html#active-tasks>>
>>>> [3]
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-system
>> <
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-system>
>> <
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-system
>> <
>> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-system
>>>>
>>
>>
