konrad-ohms commented on issue #3517:
URL: https://github.com/apache/couchdb/issues/3517#issuecomment-904550385
Thanks @nickva and sorry for my delayed response, it took me a while to
gather the right data to check if that change would be feasible for our
application code.
TLDR: The proposed solution works and is about 130 times faster on the given
dataset, that are great results, thanks so much for your input :-)
## View adjustments
I tried to use the build-in reducers for a small set of documents on CouchDB
3.
For the testing out your proposal, I created 7 documents, all shared the
same aDefId to return one single reduced output object. I have one document per
execution result and an additional document which executed successful to be
able to check the computation.
The original view was using this map function:
```javascript
function (doc) {
if(doc.object === "ainst" && doc.result) {
var subscription = doc.subscription || doc.tenant;
var stats = {
successfulCount : 0,
failedCount : 0,
canceledCount : 0,
executingCount : 0,
unsuccessfulCount : 0,
unknownCount : 0,
totalCount : 1,
execTimeSum : 0,
execTimeCount : 0,
execTimeMin : Infinity,
execTimeMax : 0
}
if (doc.result.status === "successful") {
stats.successfulCount = 1;
if (doc.result.executionTime) {
stats.execTimeSum = doc.result.executionTime;
stats.execTimeCount = 1;
stats.execTimeMin = doc.result.executionTime;
stats.execTimeMax = doc.result.executionTime;
}
} else if (doc.result.status === "failed") {
stats.failedCount = 1;
} else if (doc.result.status === "canceled") {
stats.canceledCount = 1;
} else if (doc.result.status === "executing") {
stats.executingCount = 1;
} else if (doc.result.status === "unsuccessful") {
stats.unsuccessfulCount = 1;
} else if (doc.result.status === "unknown") {
stats.unknownCount = 1;
}
emit([subscription, doc.aDefId], stats);
}
}
```
and the following custom reduce function:
```javascript
function (keys, values, rereduce) {
var stats = {
successfulCount : 0,
failedCount : 0,
canceledCount : 0,
executingCount : 0,
unsuccessfulCount : 0,
unknownCount : 0,
totalCount : 0,
execTimeSum : 0,
execTimeCount : 0,
execTimeMin : Infinity,
execTimeMax : 0
}
for (var index in values) {
stats.successfulCount += values[index].successfulCount;
stats.failedCount += values[index].failedCount;
stats.canceledCount += values[index].canceledCount;
stats.executingCount += values[index].executingCount;
stats.unsuccessfulCount += values[index].unsuccessfulCount;
stats.unknownCount += values[index].unknownCount;
stats.totalCount += values[index].totalCount;
if (values[index].execTimeCount) {
stats.execTimeSum += values[index].execTimeSum;
stats.execTimeCount += values[index].execTimeCount;
stats.execTimeMin = Math.min(stats.execTimeMin,
values[index].execTimeMin);
stats.execTimeMax = Math.max(stats.execTimeMax,
values[index].execTimeMax);
}
}
return stats;
}
```
The original view returned the following as response which I will use to
validate the result of the following adjustments:
```bash
$ curl -ksu "admin:password"
"http://localhost:3003/demo2/_design/demo/_view/slow?reduce=true&group=true"; |
jq "."
{
"rows": [
{
"key": [
"xxx",
"135600b7487db5804c1961408a115e0270e5"
],
"value": {
"successfulCount": 2,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 7,
"execTimeSum": 1557,
"execTimeCount": 2,
"execTimeMin": 200,
"execTimeMax": 1357
}
}
]
}
```
As you mentioned, the build-in _sum reducer can be used to count different
executions in a single view.
This is my updated demo-build-in-reduce/count view:
```javascript
function (doc) {
if(doc.object === "ainst" && doc.result) {
var subscription = doc.subscription || doc.tenant;
var stats = {
successfulCount : 0,
failedCount : 0,
canceledCount : 0,
executingCount : 0,
unsuccessfulCount : 0,
unknownCount : 0,
totalCount : 1
}
if (doc.result.status === "successful") {
stats.successfulCount = 1;
} else if (doc.result.status === "failed") {
stats.failedCount = 1;
} else if (doc.result.status === "canceled") {
stats.canceledCount = 1;
} else if (doc.result.status === "executing") {
stats.executingCount = 1;
} else if (doc.result.status === "unsuccessful") {
stats.unsuccessfulCount = 1;
} else if (doc.result.status === "unknown") {
stats.unknownCount = 1;
}
emit([subscription, doc.aDefId], stats);
}
}
```
The reduce function is set to _sum on that view.
When I query that view, I get results as expected for the counts:
```bash
$ curl -ksu "admin:password"
"http://localhost:3003/demo2/_design/demo-build-in-reduce/_view/count?reduce=true&group=true";
| jq "."
{
"rows": [
{
"key": [
"xxx",
"135600b7487db5804c1961408a115e0270e5"
],
"value": {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 2,
"totalCount": 7,
"unknownCount": 1,
"unsuccessfulCount": 1
}
}
]
}
```
The response looks good, but as expected is lacking the execution time
stats. This can be done with a second view which I named
demo-build-in-reduce/successfulExecutionTime.
The second view returns the following:
```bash
$ curl -ksu "admin:password"
"http://localhost:3003/demo2/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true";
| jq "."
{
"rows": [
{
"key": [
"xxx",
"135600b7487db5804c1961408a115e0270e5"
],
"value": {
"sum": 1557,
"count": 2,
"min": 200,
"max": 1357,
"sumsqr": 1881449
}
}
]
}
```
This looks good either, I do not care for the sumsqr, but for sum, count,
min and max. The count is also included in the demo-build-in-reduce/count view,
which can be used in the application theoretically to render a more high level
view of data. The successfulExecutionTime view is providing the other data.
So, your proposal works for the given data model, thank you very much. The
downside is that to keep our API compatible, we need to submit two requests
concurrently instead of using the single original view. Nevertheless, the
performance is way more consistent, let me share a few measurements based on
the new views.
## Test setup to measure performance improvements of the new implementation
I used the initial setup script to deploy both CouchDBs and to load data.
Afterwards I injected the design doc into the demo database:
```bash
$ curl -X POST \
"http://admin:password@localhost:3003/demo"; \
-H "Content-Type: application/json" \
-d '{
"_id": "_design/demo-build-in-reduce",
"views": {
"count": {
"reduce": "_sum",
"map": "function (doc) {\n if(doc.object ===
\"ainst\" && doc.result) {\n var subscription =
doc.subscription || doc.tenant;\n var stats = {\n
successfulCount : 0,\n failedCount : 0,\n
canceledCount : 0,\n executingCount
: 0,\n unsuccessfulCount : 0,\n
unknownCount : 0,\n totalCount : 1\n
}\n\n if (doc.result.status === \"successful\") {\n
stats.successfulCount = 1;\n } else if
(doc.result.status === \"failed\") {\n stats.failedCount
= 1;\n } else if (doc.result.status === \"canceled\") {\n
stats.canceledCount = 1;\n } else if
(doc.result.status === \"executing\") {\n stats.e
xecutingCount = 1;\n } else if (doc.result.status ===
\"unsuccessful\") {\n stats.unsuccessfulCount = 1;\n
} else if (doc.result.status === \"unknown\") {\n
stats.unknownCount = 1;\n }\n\n
emit([subscription, doc.aDefId], stats);\n }\n }"
},
"successfulExecutionTime": {
"reduce": "_stats",
"map": "function (doc) {\n if(doc.object === \"ainst\" &&
doc.result && doc.result.status === \"successful\") {\n var subscription
= doc.subscription || doc.tenant;\n emit([subscription, doc.aDefId],
doc.result.executionTime);\n }\n}"
}
},
"language": "javascript"
}'
{"ok":true,"id":"_design/demo-build-in-reduce","rev":"1-e769ee9d7efdeb5f80075e16a1113968"}
```
I used the following Node.js script to query data and combine them within
the application.
[reducers-demo.zip](https://github.com/apache/couchdb/files/7038700/reducers-demo.zip)
```javascript
'use strict';
const parallel = require('async').parallel;
const http = require('http');
const user = 'admin';
const password = 'password';
const port = 3003;
const dbname = 'demo';
let newRequest = (async () => {
console.log("==================== New approach ====================");
for( let i=1; i<=10; i++) {
console.log(`===== Iteration ${i} ======`);
console.time(`Fast iteration ${i} took`);
let responseMapCount = {};
let responseMapSuccessfulExecutionTime = {};
await parallel([
(cb) => {
let currentUrl =
`http://${user}:${password}@localhost:${port}/${dbname}/_design/demo-build-in-reduce/_view/count?reduce=true&group=true`;
console.log(`Requesting: ${currentUrl}`)
let req = http.get(currentUrl, (res) => {
console.log('statusCode:', res.statusCode);
let body = ""
res.on('data', (d) => {
body += d;
});
res.on('end', function () {
// store rows in map to combine them after http
requests are completed
body = JSON.parse(body);
body.rows.forEach(currentRow => {
responseMapCount[currentRow.key.join('-')] =
currentRow.value;
});
cb()
});
}).on('error', (e) => {
console.error(e);
cb(e);
});
req.end();
},
(cb) => {
let currentUrl =
`http://${user}:${password}@localhost:${port}/${dbname}/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true`;
console.log(`Requesting: ${currentUrl}`)
let req = http.get(currentUrl, (res) => {
console.log('statusCode:', res.statusCode);
let body = '';
res.on('data', (d) => {
body += d;
});
res.on('end', function () {
// store rows in map to combine them after http
requests are completed
body = JSON.parse(body);
body.rows.forEach(currentRow => {
responseMapSuccessfulExecutionTime[currentRow.key.join('-')] = currentRow.value;
});
cb()
});
}).on('error', (e) => {
console.error(e);
cb(e);
});
req.end();
}
]);
console.log(`Combining results`);
// iterate over responseMapCount, as there is always an entry for
every ainst result
let lastMergedKey;
Object.keys(responseMapCount).forEach((currentKey) => {
// check if the aInst result had a successful invocation, merge
it back in that case
if(responseMapSuccessfulExecutionTime[currentKey]) {
// merge second result property back into responseMapCount
responseMapCount[currentKey].execTimeSum =
responseMapSuccessfulExecutionTime[currentKey].sum;
responseMapCount[currentKey].execTimeMin =
responseMapSuccessfulExecutionTime[currentKey].min;
responseMapCount[currentKey].execTimeMax =
responseMapSuccessfulExecutionTime[currentKey].max;
}
});
console.log(`Example combined result for
aDefId=100200b7487db5804c1961408a115e0270e5:
${JSON.stringify(responseMapCount['xxx-100200b7487db5804c1961408a115e0270e5'],
null, 4)}`)
console.timeEnd(`Fast iteration ${i} took`);
}
});
let oldRequest = (async () => {
console.log("==================== Old approach ====================");
for( let i=1; i<=10; i++) {
console.log(`===== Iteration ${i} ======`);
console.time(`Slow iteration ${i} took`);
let results = await parallel([
(cb) => {
let currentUrl =
`http://${user}:${password}@localhost:${port}/${dbname}/_design/demo/_view/slow?reduce=true&group=true`;
console.log(`Requesting: ${currentUrl}`)
let req = http.get(currentUrl, (res) => {
console.log('statusCode:', res.statusCode);
let body = ''
res.on('data', (d) => {
body += d;
});
res.on('end', function () {
body = JSON.parse(body);
cb(null, body);
});
}).on('error', (e) => {
console.error(e);
cb(e);
});
req.end();
}
]);
console.log(`Result for
aDefId=100200b7487db5804c1961408a115e0270e5:`);
// the view could have been invoked directly with the key, but all
results should be retrieved to measure the performance
// only a single result is picked to compare those are equal in both
query approaches.
results[0].rows.forEach((currentRow) => {
if(currentRow.key[1] === '100200b7487db5804c1961408a115e0270e5')
{
console.log(JSON.stringify(currentRow.value, null, 4));
}
})
console.timeEnd(`Slow iteration ${i} took`);
}
});
(async () => {
await newRequest();
await oldRequest();
})()
```
The output shows an impressive difference, even if the application logic
gets more complicated:
```bash
==================== New approach ====================
===== Iteration 1 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 1 took: 270.495ms
===== Iteration 2 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 2 took: 262.454ms
===== Iteration 3 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 3 took: 261.309ms
===== Iteration 4 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 4 took: 271.7ms
===== Iteration 5 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 5 took: 280.972ms
===== Iteration 6 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 6 took: 263.916ms
===== Iteration 7 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 7 took: 291.969ms
===== Iteration 8 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 8 took: 308.858ms
===== Iteration 9 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 9 took: 267.336ms
===== Iteration 10 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/count?reduce=true&group=true
Requesting:
http://admin:password@localhost:3003/demo/_design/demo-build-in-reduce/_view/successfulExecutionTime?reduce=true&group=true
statusCode: 200
statusCode: 200
Combining results
Example combined result for aDefId=100200b7487db5804c1961408a115e0270e5: {
"canceledCount": 1,
"executingCount": 1,
"failedCount": 1,
"successfulCount": 1,
"totalCount": 6,
"unknownCount": 1,
"unsuccessfulCount": 1,
"execTimeSum": 1002,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Fast iteration 10 took: 259.752ms
==================== Old approach ====================
===== Iteration 1 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 1 took: 34.973s
===== Iteration 2 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 2 took: 35.636s
===== Iteration 3 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 3 took: 35.189s
===== Iteration 4 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 4 took: 37.665s
===== Iteration 5 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 5 took: 37.669s
===== Iteration 6 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 6 took: 35.679s
===== Iteration 7 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 7 took: 35.368s
===== Iteration 8 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 8 took: 34.711s
===== Iteration 9 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 9 took: 34.831s
===== Iteration 10 ======
Requesting:
http://admin:password@localhost:3003/demo/_design/demo/_view/slow?reduce=true&group=true
statusCode: 200
Result for aDefId=100200b7487db5804c1961408a115e0270e5:
{
"successfulCount": 1,
"failedCount": 1,
"canceledCount": 1,
"executingCount": 1,
"unsuccessfulCount": 1,
"unknownCount": 1,
"totalCount": 6,
"execTimeSum": 1002,
"execTimeCount": 1,
"execTimeMin": 1002,
"execTimeMax": 1002
}
Slow iteration 10 took: 35.418s
```
## Conclusion
On average (10 iterations), I could get combined results of all records
within 274 ms compared to 35714 ms, so the workaround on my given dataset is
actually ~ 130 times faster.
I repeated the test for CouchDB 2 as well and got an average of 447 ms with
the new approach compared to 6917 ms which is still more than 15 times faster.
So independent on the question if we should migrate to CouchDB 3 or not, it
might still be a useful optimization also for CouchDB 2 setups.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
