luca-simonetti commented on issue #5044:
URL: https://github.com/apache/couchdb/issues/5044#issuecomment-2101928684
I don't think the disk is the issue here:
```
procs -----------memory---------- ---swap-- -----io---- -system--
------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id
wa st
2 0 0 312512 478476 2741660 0 0 2 19 0 0 3 0 97
0 0
```
as you can see the system doesn't spend any time waiting for IO ops.
What I see though is a memory cache increasing. After restart that very same
cache decreases a lot and starts growing back again.
One more note: the requests that are slow are not the
"include_documents=true" ones, but the "attachments=true"
I reiterate just so we're on the same page: we do 10-20 millions request to
download ALL documents with a key prefix in the database. For example we might
have
piece0000001
piece0000002
...
piece1000000
...
piece9999999
and we make a request for each of those. So if there's something cached it's
of no use or even in this case a issue since each subsequent request is
accessing a different document from the request before.
Also: we cannot change the Q factor since the database is already created
and we don't want to risk resharding and losing the whole thing...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
