Hi Simon, I wouldn't skip the deleted documents, a deleted doc could just as
easily be the one missing in the ID index. When you lookup a deleted document
you should see "reason":"deleted" instead of the "reason":"missing" that you
get if the ID is not in the index at all. Then again, if you see that only
ever see deleted docs after retrieving the full _changes then a deleted doc is
probably not the source of your troubles.
Can you confirm that the number of rows in the changes feed is equal to
doc_count + doc_del_count from db.info()? Best,
Adam
On Aug 11, 2011, at 12:08 PM, Simon Eisenmann wrote:
> Hi Adam,
>
> i wrote a short python script, which loads the complete changes feed and
> requests all of the documents not marked "deleted" using HTTP HEAD
> requests. Though i only find documents which have been deleted after i
> have retrieved the complete changes view (takes a while for large
> databases), and never the same.
>
> So again no luck here in finding the problems.
>
> Any more suggestions? Is there a way to rebuild the complete database
> file (maybe offline?).
>
> Thank you and best regards
> Simon
>
>
> ps: The script i have been using is attached.
>
>
>
> Am Dienstag, den 09.08.2011, 16:10 -0400 schrieb Adam Kocoloski:
>> Hi Simon, CouchDB 1.1.0 includes a recent optimization to
>> _changes?include_docs=true which allows it to skip a lookup in the id tree
>> and instead load the document body from the pointer in the sequence tree.
>> In that case you wouldn't notice any missing entry in the id tree. You
>> would notice it, however, if you did direct lookups for each document.
>> Apologies for the outdated instructions. Can you try looking up the
>> documents in a separate request and see if the results change?
>>
>> Adam
>>
>> On Aug 9, 2011, at 5:49 AM, Simon Eisenmann wrote:
>>
>>> Hi Adam,
>>>
>>> i just checked the whole _changes feed (since=0) and could not find any
>>> document "missing" when using "include_docs=true".
>>>
>>> The database in itself has only around 30 documents, so should be quite
>>> small. Though there are lots of creations and deletions happening all
>>> the time. Thus it its daily purged and compacted.
>>>
>>> So - the changes feed is of no help. Any other idea?
>>>
>>> Thank you and best regards
>>> Simon
>>>
>>> Am Montag, den 08.08.2011, 13:25 -0400 schrieb Adam Kocoloski:
>>>> Hi Simon, I think my amended instructions to Mike are still a sensible way
>>>> to debug/workaround the problem. Reiterating (96282148 was the last seq
>>>> Mike observed in the Futon status for the compaction):
>>>>
>>>>> 1) What you really want are the last 1000 Ids in the seq_tree prior to
>>>>> the compactor crash. So maybe something like
>>>>>
>>>>> GET /iris/_changes?descending=true&limit=1000&since=96282148
>>>>
>>>>> 2) Figure out which of those entries are missing from the id tree, e.g.
>>>>> lookup the document and see if the response is {"not_found":"missing"}.
>>>>> You could also try using include_docs=true on the _changes feed to
>>>>> accomplish the same.
>>>>
>>>>> 3) Once you've identified the problematic IDs, try creating them again.
>>>>> You might end up introducing duplicates in the _changes feed, but if you
>>>>> do there's a procedure to fix that.
>>>>
>>>> Regards, Adam
>>>>
>>>> On Aug 8, 2011, at 12:31 PM, Simon Eisenmann wrote:
>>>>
>>>>> Hi Guys,
>>>>>
>>>>> i have a couple of CouchDB instances which started to be come
>>>>> unpackable. It shows this error:
>>>>>
>>>>> [Mon, 08 Aug 2011 16:16:27 GMT] [info] [<0.10808.123>] Starting
>>>>> compaction for db "database1"
>>>>> [Mon, 08 Aug 2011 16:16:45 GMT] [error] [<0.10808.123>] ** Generic
>>>>> server <0.10808.123> terminating
>>>>> ** Last message in was {'EXIT',<0.30396.143>,
>>>>> {function_clause,
>>>>> [{couch_db_updater,'-copy_docs/4-fun-2-',
>>>>> [not_found,
>>>>> {db,<0.10807.123>,<0.10808.123>,nil,
>>>>> <<"1312767347007568">>,<0.10805.123>,
>>>>> <0.10809.123>,
>>>>> {db_header,5,6340261,0,
>>>>> {7198006895,{65952,10145}},
>>>>> {7198010315,76813},
>>>>> {7198051016,[]},
>>>>> 364,7050915618,nil,1000},
>>>>> 6340261,
>>>>> {btree,<0.10805.123>,
>>>>> {7198006895,{65952,10145}},
>>>>> #Fun<couch_db_updater.10.19222179>,
>>>>> #Fun<couch_db_updater.11.21515767>,
>>>>> #Fun<couch_btree.5.124754102>,
>>>>> #Fun<couch_db_updater.12.93888648>},
>>>>> {btree,<0.10805.123>,
>>>>> {7198010315,76813},
>>>>> #Fun<couch_db_updater.13.40165027>,
>>>>> #Fun<couch_db_updater.14.82810239>,
>>>>> #Fun<couch_btree.5.124754102>,
>>>>> #Fun<couch_db_updater.15.104121193>},
>>>>> {btree,<0.10805.123>,
>>>>> {7198051016,[]},
>>>>> #Fun<couch_btree.0.83553141>,
>>>>> #Fun<couch_btree.1.30790806>,
>>>>> #Fun<couch_btree.2.124754102>,nil},
>>>>> 6340261,<<"spreedcom_accounts_1">>,
>>>>> "/var/lib/couchdb/database1.couch",[],
>>>>> [],nil,
>>>>> {user_ctx,null,[],undefined},
>>>>> nil,1000,
>>>>> [before_header,after_header,on_file_open],
>>>>> false},
>>>>> <0.30397.143>]},
>>>>> {lists,map,2},
>>>>> {lists,map,2},
>>>>> {couch_db_updater,copy_docs,4},
>>>>> {couch_db_updater,'-copy_compact/3-fun-0-',6},
>>>>> {couch_btree,stream_kv_node2,8},
>>>>> {couch_btree,stream_kp_node,7},
>>>>> {couch_btree,fold,4}]}}
>>>>>
>>>>>
>>>>> ... lots of more similar errors following.
>>>>>
>>>>> In this mailing list i have found a similar issue from the beginning of
>>>>> this year (Fri, 31 Dec 2010 12:38:18), though without a solution.
>>>>>
>>>>> This database got purged a lot and has constant changes. So pretty
>>>>> similar from the older topic. So i assume there is something wrong in
>>>>> the database file related to previous purges. So looks like some bug
>>>>> here.
>>>>>
>>>>> So - any hints how to fix this? The database is getting pretty large and
>>>>> has to be packed from time to time.
>>>>>
>>>>> CouchDB is running with Version 1.1.0 on Linux 64bit. The database has
>>>>> initially been created with CouchDB 1.0.1 - though the issue appeared a
>>>>> couple of weeks ago (packing has been working with 1.1.0 before.
>>>>>
>>>>>
>>>>> Thank you and best regards
>>>>> Simon
>>>>>
>>>>>
>>>>> --
>>>>> Simon Eisenmann
>>>>>
>>>>> [ mailto:[email protected] ]
>>>>>
>>>>> [ struktur AG | Kronenstraße 22a | D-70173 Stuttgart ]
>>>>> [ T. +49.711.896656.0 | F.+49.711.89665610 ]
>>>>> [ http://www.struktur.de | mailto:[email protected] ]
>>>>
>>>
>>> --
>>> Simon Eisenmann
>>>
>>> [ mailto:[email protected] ]
>>>
>>> [ struktur AG | Kronenstraße 22a | D-70173 Stuttgart ]
>>> [ T. +49.711.896656.68 | F.+49.711.89665610 ]
>>> [ http://www.struktur.de | mailto:[email protected] ]
>>
>
> --
> Simon Eisenmann
>
> [ mailto:[email protected] ]
>
> [ struktur AG | Kronenstraße 22a | D-70173 Stuttgart ]
> [ T. +49.711.896656.68 | F.+49.711.89665610 ]
> [ http://www.struktur.de | mailto:[email protected] ]
> <repaircompact.py>
