evsukov89 opened a new issue #1151: {db}/_changes?filter=_selector lags towards
almost the end of the response
URL: https://github.com/apache/couchdb/issues/1151
While benchmarking the performance of mango selectors I stumbled upon an
interesting behaviour: towards almost the end of the response, where there's 3
- 5 - 10 more changes expected to be received the response pauses from 0.5 sec
to more, depending on the doc_count in the db, then finishes as normal.
## Expected Behavior
No pause/lag in the _changes feed response
## Current Behavior
Consistent one-time pause/lag towards the end of the response. The bigger
the db ??the bigger the pause.
<!--- If describing a bug, tell us what happens instead of the expected
behavior -->
<!--- If suggesting a change/improvement, explain the difference from
current behavior -->
## Possible Solution
Cannot suggest much at the moment, still looking into the code.
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
1. run the latest version from the official Docker container `docker run -p
5984:5984 -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password --name couchdb
apache/couchdb`
2. configure as a single node
3. generate a db using the following script
https://gist.github.com/evsukov89/0101dd1fcea728ae6c5807eef70a7820
4. query the _changes feed `curl -s -u admin:password -H "Content-Type:
application/json" -X POST
'http://127.0.0.1:5984/db-1000/_changes?filter=_selector' --data
'{"selector":{"owner":"global"}}'`
5. observe the lag towards the end of the response, see example video
https://www.dropbox.com/s/jc99cnyytznbmt6/couchdb2_filter_selector_lag.mov?dl=0
Change the `NUMGLOBAL` variable in the script to create more/less documents
and experiment with different database sizes.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful
in the real world -->
I'm investigating mango selectors for replication as an alternative to view
_change which appears not to be functional in 2.x (see #831). In my use-case, I
operate over a multi-tenant environment (think one couch database ? one
application instance/project/customer/etc.), a subset of documents, typically
around 10% is being replicated by mobile clients. Having a performant way to
get a filtered _changes feed is a must.
While it's not really a bug nor a blocker in my case, the delay present 100%
during my tests and is proportional to the number of docs in the db; I would
simply like to understand the root cause and potential performance implications
when having 100s of DBs on the same cluster with 10s of thousands of mobile
clients replicating over with typical db size of 50-80k docs.
## Your Environment
<!--- Include as many relevant details about the environment you experienced
the bug in -->
* Version used:
```
Docker version 17.12.0-ce, build c97c6d6
```
```
{"couchdb":"Welcome","version":"2.1.1","features":["scheduler"],"vendor":{"name":"The
Apache Software Foundation"}}
```
* Browser Name and version:
```
curl 7.54.0 (x86_64-apple-darwin17.0) libcurl/7.54.0 LibreSSL/2.0.20
zlib/1.2.11 nghttp2/1.24.0
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3
pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL
libz HTTP2 UnixSockets HTTPS-proxy
```
* Operating System and version (desktop or mobile):
```
ProductName: Mac OS X
ProductVersion: 10.13.3
BuildVersion: 17D47
```
* Link to your project: n/a
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
