tl;dir: this doesn’t look good, and shouldn’t happen from what CouchDB is doing to the file. Maybe something external messed with the DB file.
Could you share the fist megabyte of the file somewhere? What filesystem is this on? Did the machine reboot at any time between the creation of the database and now? Did the CouchDB process terminate (intentionally or unintentionally at any time between the creation of the database and now? Did you backup/restore the hard drive at any time between the creation of the database and now? Did you update the CouchDB server at any time between the creation of the database and now? If yes, from which version? What is your “delayed_commits” setting? Best, Jan -- Professional Support for Apache CouchDB: http://www.neighbourhood.ie/couchdb-support/ > On 14 Mar 2015, at 08:51, Frank Röhm <francwal...@gmx.net> wrote: > > Hello > > On my Ubuntu 12.04 LTS server I have installed couchdb version 1.6.1-0ubuntu1. > $ couchdb -V > Apache CouchDB 1.6.1 > > I have “scraped” the german wiki into this couchdb and now I have a > uncompressed 44 GB database “de-m-wikipedia-org”, which I try to compact with: > > $ curl -H "Content-Type: application/json" -X POST > http://localhost:5984/de-m-wikipedia-org/_compact > {"ok":true} > > But this throws an error: > > Error in process <0.247.0> with exit value: > {function_clause,[{couch_compress,decompress,[<<0 > bytes>>]},{couch_file,pread_term,2},{couch_btree,get_node,2},{couch_btree,stream_node,7},{couch_btree,stream_kp_node,7},{couch_btree,stream_kp_node,8},{couch_btree,fold,4},{couch_db_updater... > > complete error is attached in the end of this mail. > > and I get a file de-m-wikipedia-org.couch.compact which has only 79 Byte and > wouldn’t grow bigger. > > If I compact another db, which had 7.4 GB, the compaction works flawless, I > read in the log: > > [Sat, 14 Mar 2015 07:37:21 GMT] [info] [<0.695.0>] Starting compaction for db > "de-m-wiktionary-org" > [Sat, 14 Mar 2015 07:37:21 GMT] [info] [<0.139.0>] 127.0.0.1 - - POST > /de-m-wiktionary-org/_compact 202 > [Sat, 14 Mar 2015 07:39:39 GMT] [info] [<0.695.0>] Compaction for db > "de-m-wiktionary-org" completed. > > and after compaction the file is 1.7 GB. > > So I guess it is related to this big database. > Should I just delete it and restart to scrape again from scratch? > > > Thank you > > frank > > > > > > > PS.: this is the complete error message in the couch.log > > > [Sat, 14 Mar 2015 07:28:35 GMT] [info] [<0.31.0>] Apache CouchDB has started > on http://127.0.0.1:5984/ > [Sat, 14 Mar 2015 07:30:05 GMT] [info] [<0.242.0>] Starting compaction for db > "de-m-wikipedia-org" > [Sat, 14 Mar 2015 07:30:05 GMT] [info] [<0.137.0>] 127.0.0.1 - - POST > /de-m-wikipedia-org/_compact 202 > [Sat, 14 Mar 2015 07:30:05 GMT] [error] [emulator] Error in process <0.247.0> > with exit value: {function_clause,[{couch_compress,decompress,[<<0 > bytes>>]},{couch_file,pread_term,2},{couch_btree,get_node,2},{couch_btree,stream_node,7},{couch_btree,stream_kp_node,7},{couch_btree,stream_kp_node,8},{couch_btree,fold,4},{couch_db_updater... > > > [Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.242.0>] ** Generic server > <0.242.0> terminating > ** Last message in was {'EXIT',<0.247.0>, > {function_clause, > [{couch_compress,decompress,[<<>>]}, > {couch_file,pread_term,2}, > {couch_btree,get_node,2}, > {couch_btree,stream_node,7}, > {couch_btree,stream_kp_node,7}, > {couch_btree,stream_kp_node,8}, > {couch_btree,fold,4}, > {couch_db_updater,copy_compact,3}]}} > ** When Server state == {db,<0.241.0>,<0.242.0>,<0.247.0>, > <<"1426318205344581">>,<0.243.0>,<0.239.0>, > <0.245.0>, > {db_header,6,2624284,0, > {46859286084, > {1775313,0,15923938389}, > 159502872}, > {46859291358,1775313,123250096}, > nil,0,nil,nil,1000}, > 2624284, > {btree,<0.239.0>, > {46859286084, > {1775313,0,15923938389}, > 159502872}, > #Fun<couch_db_updater.10.55895019>, > #Fun<couch_db_updater.11.100913286>, > #Fun<couch_btree.5.296538>, > #Fun<couch_db_updater.12.39068440>,snappy}, > {btree,<0.239.0>, > {46859291358,1775313,123250096}, > #Fun<couch_db_updater.13.114276184>, > #Fun<couch_db_updater.14.2340873>, > #Fun<couch_btree.5.296538>, > #Fun<couch_db_updater.15.23651859>,snappy}, > {btree,<0.239.0>,nil, > #Fun<couch_btree.3.127739064>, > #Fun<couch_btree.4.112272954>, > #Fun<couch_btree.5.296538>,nil,snappy}, > 2624284,<<"de-m-wikipedia-org">>, > "/var/lib/couchdb/de-m-wikipedia-org.couch",[],[], > nil, > {user_ctx,null,[],undefined}, > nil,1000, > [before_header,after_header,on_file_open], > [{user_ctx, > {user_ctx,null, > [<<"_admin">>], > <<"{couch_httpd_auth, > default_authentication_handler}">>}}], > snappy,nil,nil} > ** Reason for termination == > ** {function_clause,[{couch_compress,decompress,[<<>>]}, > {couch_file,pread_term,2}, > {couch_btree,get_node,2}, > {couch_btree,stream_node,7}, > {couch_btree,stream_kp_node,7}, > {couch_btree,stream_kp_node,8}, > {couch_btree,fold,4}, > {couch_db_updater,copy_compact,3}]} > > [Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.242.0>] {error_report,<0.30.0>, > {<0.242.0>,crash_report, > [[{initial_call, > {couch_db_updater,init,['Argument__1']}}, > {pid,<0.242.0>}, > {registered_name,[]}, > {error_info, > {exit, > {function_clause, > [{couch_compress,decompress,[<<>>]}, > {couch_file,pread_term,2}, > {couch_btree,get_node,2}, > {couch_btree,stream_node,7}, > {couch_btree,stream_kp_node,7}, > {couch_btree,stream_kp_node,8}, > {couch_btree,fold,4}, > {couch_db_updater,copy_compact,3}]}, > [{gen_server,terminate,6}, > {proc_lib,init_p_do_apply,3}]}}, > {ancestors,[<0.241.0>,<0.238.0>]}, > {messages,[]}, > {links,[<0.241.0>]}, > {dictionary,[]}, > {trap_exit,true}, > {status,running}, > {heap_size,987}, > {stack_size,24}, > {reductions,1753}], > []]}} > [Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.241.0>] ** Generic server > <0.241.0> terminating > ** Last message in was {'EXIT',<0.242.0>, > {function_clause, > [{couch_compress,decompress,[<<>>]}, > {couch_file,pread_term,2}, > {couch_btree,get_node,2}, > {couch_btree,stream_node,7}, > {couch_btree,stream_kp_node,7}, > {couch_btree,stream_kp_node,8}, > {couch_btree,fold,4}, > {couch_db_updater,copy_compact,3}]}} > ** When Server state == {db,<0.241.0>,<0.242.0>,<0.247.0>, > <<"1426318205344581">>,<0.243.0>,<0.239.0>, > <0.245.0>, > {db_header,6,2624284,0, > {46859286084, > {1775313,0,15923938389}, > 159502872}, > {46859291358,1775313,123250096}, > nil,0,nil,nil,1000}, > 2624284, > {btree,<0.239.0>, > {46859286084, > {1775313,0,15923938389}, > 159502872}, > #Fun<couch_db_updater.10.55895019>, > #Fun<couch_db_updater.11.100913286>, > #Fun<couch_btree.5.296538>, > #Fun<couch_db_updater.12.39068440>,snappy}, > {btree,<0.239.0>, > {46859291358,1775313,123250096}, > #Fun<couch_db_updater.13.114276184>, > #Fun<couch_db_updater.14.2340873>, > #Fun<couch_btree.5.296538>, > #Fun<couch_db_updater.15.23651859>,snappy}, > {btree,<0.239.0>,nil, > #Fun<couch_btree.3.127739064>, > #Fun<couch_btree.4.112272954>, > #Fun<couch_btree.5.296538>,nil,snappy}, > 2624284,<<"de-m-wikipedia-org">>, > "/var/lib/couchdb/de-m-wikipedia-org.couch",[],[], > nil, > {user_ctx,null,[],undefined}, > nil,1000, > [before_header,after_header,on_file_open], > [{user_ctx, > {user_ctx,null, > [<<"_admin">>], > <<"{couch_httpd_auth, > default_authentication_handler}">>}}], > snappy,nil,nil} > ** Reason for termination == > ** {function_clause,[{couch_compress,decompress,[<<>>]}, > {couch_file,pread_term,2}, > {couch_btree,get_node,2}, > {couch_btree,stream_node,7}, > {couch_btree,stream_kp_node,7}, > {couch_btree,stream_kp_node,8}, > {couch_btree,fold,4}, > {couch_db_updater,copy_compact,3}]} > > [Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.241.0>] {error_report,<0.30.0>, > {<0.241.0>,crash_report, > [[{initial_call,{couch_db,init,['Argument__1']}}, > {pid,<0.241.0>}, > {registered_name,[]}, > {error_info, > {exit, > {function_clause, > [{couch_compress,decompress,[<<>>]}, > {couch_file,pread_term,2}, > {couch_btree,get_node,2}, > {couch_btree,stream_node,7}, > {couch_btree,stream_kp_node,7}, > {couch_btree,stream_kp_node,8}, > {couch_btree,fold,4}, > {couch_db_updater,copy_compact,3}]}, > [{gen_server,terminate,6}, > {proc_lib,init_p_do_apply,3}]}}, > {ancestors,[<0.238.0>]}, > {messages,[]}, > {links,[<0.88.0>]}, > {dictionary,[]}, > {trap_exit,true}, > {status,running}, > {heap_size,987}, > {stack_size,24}, > {reductions,284}], > []]}} > [Sat, 14 Mar 2015 07:30:05 GMT] [error] [<0.88.0>] Unexpected exit of > database process <0.241.0> [<<"de-m-wikipedia-org">>]: {function_clause, > > [{couch_compress, > > decompress, > > [<<>>]}, > > {couch_file, > > pread_term, > > 2}, > > {couch_btree, > > get_node, > > 2}, > > {couch_btree, > > stream_node, > > 7}, > > {couch_btree, > > stream_kp_node, > > 7}, > > {couch_btree, > > stream_kp_node, > > 8}, > > {couch_btree, > > fold, > > 4}, > > {couch_db_updater, > > copy_compact, > > 3}]} > >
signature.asc
Description: Message signed with OpenPGP using GPGMail
