byrneml opened a new issue #3594: URL: https://github.com/apache/couchdb/issues/3594
[NOTE]: # ( ^^ Provide a general summary of the issue in the title above. ^^ ) ## Description Continuous replication of https://skimdb.npmjs.com/registry gets extremely slow over time - it worked fine for up to 400k documents, then became extremely slow. The entire dataset contains 1.6M documents. **Here is the active task** ``` [ { "node": "couchdb@localhost", "pid": "<0.1300.89>", "changes_pending": 13442245, "checkpoint_interval": 30000, "checkpointed_source_seq": 611203, "continuous": true, "database": "shards/80000000-ffffffff/_replicator.1622573219", "doc_id": "ad71dbf4189be812bfb41c30d6000da6", "doc_write_failures": 1, "docs_read": 533605, "docs_written": 533604, "missing_revisions_found": 533605, "replication_id": "a5df5f8f23b0d94f28a1ed02d4a2f103+continuous", "revisions_checked": 540275, "source": "https://skimdb.npmjs.com/registry/";, "source_seq": 612872, "started_on": 1622581539, "target": "http://localhost:5984/npm_mirror/";, "through_seq": 612872, "type": "replication", "updated_on": 1622581842, "user": "admin" }, { "node": "couchdb@localhost", "pid": "<0.22157.91>", "changes_done": 6107, "database": "shards/00000000-7fffffff/npm_mirror.1622573154", "phase": "document_copy", "progress": 2, "started_on": 1622581839, "total_changes": 268814, "type": "database_compaction", "updated_on": 1622581841 } ] ``` ## Steps to Reproduce Using the Couchbase GUI: 1. Navigate to replication 2. Set the remote source as https://skimdb.npmjs.com/registry 3. Set the remote as a new local database ## Expected Behaviour For the replication to run more quickly than it does (3+ hours to get through ~30% of the replication). ## Your Environment We're running 1 central instance locally for now. ``` { "couchdb": "Welcome", "version": "3.1.1", "git_sha": "ce596c65d", "uuid": "5e70eea19ba38ca970bd9a8020677dbf", "features": [ "access-ready", "partitioned", "pluggable-storage-engines", "reshard", "scheduler" ], "vendor": { "name": "The Apache Software Foundation" } } ``` [TIP]: # ( You can paste the output of curl http://YOUR-COUCHDB:5984/ here. ) * CouchDB version used: * Browser name and version: * Operating system and version: ## Additional Context We have no attachments, just JSON documents. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
