Steven Schaefer created COUCHDB-2886:
----------------------------------------
Summary: Concurrent Bulk Updates and Attachments yield Corruption
Key: COUCHDB-2886
URL: https://issues.apache.org/jira/browse/COUCHDB-2886
Project: CouchDB
Issue Type: Bug
Reporter: Steven Schaefer
I work for Nexidia, and we're evaluating CouchDB for a new system we're
developing. I was running some tests and in general trying to push it as hard
as I could to help gain experience and set expectations, and I've encountered
what appears to be a database corruption issue, when conflicting concurrent
batch updates include attachments. I noticed results from one of my views made
no sense, and after verifying it was up-to-date, tried running view compaction
in an attempt to address that problem. It errored with this:
{quote}
[info] [<0.24269.24>] Compaction started for db: bug idx: _design/dev
[info] [<0.24224.24>] 127.0.0.1 - - POST /bug/_compact/dev 202
[error] [<0.24269.24>] Duplicate docid `9255f05ad2939b5c487209321c000c79`
detected in view group `_design/dev`, database `bug` - This view needs to be
rebuilt.
[error] [<0.24251.24>] ** Generic server <0.24251.24> terminating
** Last message in was {'EXIT',<0.24269.24>,
{view_duplicate_id,
<<"9255f05ad2939b5c487209321c000c79">>}}
** When Server state == {st,<0.24244.24>,couch_mrview_index,<0.24269.24>}
** Reason for termination ==
** unknown_info
{quote}
Deleting and then rebuilding the view didn't help, after which view results
were still incorrect, and re-running view compaction produced the same error,
which suggests to me a more fundamental issue.
I narrowed it down to a combination of concurrent bulk updates with
attachments, or at least I should say I can consistently reproduce when
including attachments, but I could never reproduce without them. I have
attached a smallish example that usually reproduces the issue on my system.
Notes for running:
* Depends on Java 8 and if compiling I set this up to use maven
* Program will create "bug" database in a Couch instance running at
localhost:5984. An existing "bug" db will be deleted.
* Some variables may need to be altered depending on system specs, look for
"NOTE:" in the code. Rebuilding should be as easy as (assuming maven is
installed) running from wherever unzipped: mvn install
* run-test.bat provides the command to run it. Only adjusting the -cp parameter
should be necessary for Linux (i.e. ';' --> ':' )
* Should run fairly quickly, after which "Bug reproduced" should be visible in
the console, and the Couch log will have an error like above.
System Notes:
* Windows 7
* 2x Intel Xeon E5-2687W v3 @ 3.10 GHz - 10 cores + hyperthreading (40 logical
processors)
* 32 GB ram (NUMA)
* CouchDB configuration is default except for db and view storage locations.
* CouchDB 1.6.1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
