Hi, Was the compaction finished when you tried to insert the document?
CGS On Mon, Feb 20, 2012 at 10:16 PM, Szabo, Viktor < [email protected]> wrote: > Hi, > > An application we have is using bulk writes to insert objects into the > database, and occasionally some > documents get lost. It turns out this happens for the very same reason > described below. If we're > re-insering a doc using an id that has been deleted earlier, if the > contents match the payload that was > saved when the id got used for the first time, the doc remains in > "deleted" state and the client > gets a misleading response. This only happens when the database is > compacted after the delete operation > is executed. > > I still feel that this is pretty nasty - what do you guys think? > > Looking at the reply coming from _bulk_docs, there's no way I can tell if > I can read back my doc from the database or not. > > Below are the steps to reproduce. > > Thanks, > Viktor > > $ curl -X POST -H "Content-type: application/json" > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug", "key" > : "value" } ] }' > [{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}] > > $ curl -X GET http://server/database/bug > {"_id":"bug","_rev":"1-59414e77c768bc202142ac82c2f129de","key":"value"} > > $ curl -X DELETE > http://server/database/bug?rev=1-59414e77c768bc202142ac82c2f129de > {"ok":true,"id":"bug","rev":"2-9b2e3bcc3752a3a952a3570b2ed4d27e"} > > $ curl -X GET http://server/database/bug > {"error":"not_found","reason":"deleted"} > > $ curl -X POST -H "Content-type: application/json" > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug", "key" > : "value" } ] }' > [{"id":"bug","rev":"3-82b9390af5b7c003e03c0dd7e6aac45a"}] > > $ curl -X GET http://server/database/bug > {"_id":"bug","_rev":"3-82b9390af5b7c003e03c0dd7e6aac45a","key":"value"} > > $ curl -X DELETE > http://server/database/bug?rev=3-82b9390af5b7c003e03c0dd7e6aac45a > {"ok":true,"id":"bug","rev":"4-1f98d35af7fd5ce66197980f295e5dba"} > > $ curl -X POST -H "Content-type: application/json" > http://server/database/_compact > {"ok":true} > > $ curl -X GET http://server/database/bug > {"error":"not_found","reason":"deleted"} > > $ curl -X POST -H "Content-type: application/json" > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug", "key" > : "value" } ] }' > [{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}] > > $ curl -X GET http://server/database/bug > {"error":"not_found","reason":"deleted"} > > > -----Original Message----- > From: Marcello Nuccio [mailto:[email protected]] > Sent: 23 November 2011 14:38 > To: [email protected] > Subject: Re: possible compact bug in 1.1.1 > > When a new document is created, the response is 201, not 200. > If the posted document is identical to the one saved, the revision is the > same (otherwise it would have been a conflict). So the revision you have is > not obsolete. > > Marcello > > 2011/11/23 Szabo, Viktor (Enterprise Infrastructure) > <[email protected]>: > > I'd rather know about the fact that I haven't just successfully > > created a doc, but re-submitted a revision that was already known - > > and is already obsolete as revisions with higher version numbers already > exist. > > > > Cheers, > > Viktor > > > > -----Original Message----- > > From: Marcello Nuccio [mailto:[email protected]] > > Sent: 23 November 2011 13:41 > > To: [email protected] > > Subject: Re: possible compact bug in 1.1.1 > > > > Can you elaborate on why you want a conflict? > > I find it confusing to have a conflict when, in fact, there can't be any > conflict since nothing has changed. > > > > Marcello > > > > 2011/11/23 Szabo, Viktor (Enterprise Infrastructure) > > <[email protected]>: > >> Thanks Paul, this makes sense. > >> > >> If it counts, I vote for forcing a conflict ;) > >> > >> Cheers, > >> Viktor > >> > >> -----Original Message----- > >> From: Paul Davis [mailto:[email protected]] > >> Sent: 22 November 2011 20:54 > >> To: [email protected] > >> Subject: Re: possible compact bug in 1.1.1 > >> > >> > >> Your example here is actually hitting a very specific edge case as > demonstrated by Marcello's test. As of many versions ago, revisions are > generated using a hashing scheme of the document contents. In your > particular case the requests you're issuing contain the same identical data > in such a way that CouchDB will generate a revision of the doc. > >> > >> Given this, we then have to look at how this plays into replication. > >> Basically, when we merge the revision trees we get to the case where > it's "oh, we already have this version, cool" because we do already have > this version. > >> > >> Whether or not that behavior is best, or if we should force a conflict > if we don't add a leaf during a write is another question. In other words, > the system is working fine, but this particular behavior can be a bit > unexpected. > >> > >> --------------------------------------------------------------------- > >> - > >> ---- > >> NOTICE: Morgan Stanley is not acting as a municipal advisor and the > opinions or views contained herein are not intended to be, and do not > constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall > Street Reform and Consumer Protection Act. If you have received this > communication in error, please destroy all electronic and paper copies and > notify the sender immediately. Mistransmission is not intended to waive > confidentiality or privilege. Morgan Stanley reserves the right, to the > extent permitted under applicable law, to monitor electronic > communications. This message is subject to terms available at the following > link: http://www.morganstanley.com/disclaimers. If you cannot access > these links, please notify us by reply message and we will send the > contents to you. By messaging with Morgan Stanley you consent to the > foregoing. > > > > ---------------------------------------------------------------------- > > ---- > > NOTICE: Morgan Stanley is not acting as a municipal advisor and the > opinions or views contained herein are not intended to be, and do not > constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall > Street Reform and Consumer Protection Act. If you have received this > communication in error, please destroy all electronic and paper copies and > notify the sender immediately. Mistransmission is not intended to waive > confidentiality or privilege. Morgan Stanley reserves the right, to the > extent permitted under applicable law, to monitor electronic > communications. This message is subject to terms available at the following > link: http://www.morganstanley.com/disclaimers. If you cannot access > these links, please notify us by reply message and we will send the > contents to you. By messaging with Morgan Stanley you consent to the > foregoing. > > -------------------------------------------------------------------------- > NOTICE: Morgan Stanley is not acting as a municipal advisor and the > opinions or views contained herein are not intended to be, and do not > constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall > Street Reform and Consumer Protection Act. If you have received this > communication in error, please destroy all electronic and paper copies and > notify the sender immediately. Mistransmission is not intended to waive > confidentiality or privilege. Morgan Stanley reserves the right, to the > extent permitted under applicable law, to monitor electronic > communications. This message is subject to terms available at the following > link: http://www.morganstanley.com/disclaimers. If you cannot access > these links, please notify us by reply message and we will send the > contents to you. By messaging with Morgan Stanley you consent to the > foregoing. >
