Server terminated if timeout reached during reopen_db after compaction
----------------------------------------------------------------------
Key: COUCHDB-1187
URL: https://issues.apache.org/jira/browse/COUCHDB-1187
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 1.0.2
Environment: Gentoo Linux
Erlang 14.2.1
Reporter: Alexey Loshkarev
I have installed couchdb database onto ReiserFS partition. Before databases
becomes large, all was fine.
But now, I have some databases with very much revisions in it (but very few
documents). So, it grows from 10M to 1G in couple of days. Next, i compact it
and it become 10M again.
But, if i have forgot to compact it in a few days, it becomes 10GB+. Deleting
such big files is very time-consumption on reiserfs (i know, reiser4, ext3/4 or
xfs has no such problem, but reiserfs3 has). It may take up to couple of
minutes to finish deletion.
Now, after compaction, database terminated, due timeout on reopen_db action.
Here are log:
========================
[Sun, 05 Jun 2011 14:47:16 GMT] [info] [<0.128.0>] Starting compaction for db
"exhaust"
[Sun, 05 Jun 2011 14:47:16 GMT] [info] [<0.2451.0>] 10.20.21.1 - - 'POST'
/exhaust/_compact 202
[Sun, 05 Jun 2011 14:47:21 GMT] [info] [<0.2452.0>] 10.20.21.1 - - 'GET'
/_active_tasks 200
[Sun, 05 Jun 2011 14:47:21 GMT] [info] [<0.2463.0>] 10.20.21.1 - - 'GET' / 200
[Sun, 05 Jun 2011 14:47:24 GMT] [info] [<0.128.0>] Compaction for db "exhaust"
completed.
[Sun, 05 Jun 2011 14:47:24 GMT] [info] [<0.2464.0>] 127.0.0.1 - - 'GET' / 200
[Sun, 05 Jun 2011 14:47:26 GMT] [info] [<0.2473.0>] 10.20.21.1 - - 'GET'
/_active_tasks 200
[Sun, 05 Jun 2011 14:47:26 GMT] [info] [<0.2476.0>] 10.20.21.1 - - 'GET' / 200
[Sun, 05 Jun 2011 14:47:29 GMT] [error] [<0.507.0>] ** Generic server <0.507.0>
terminating
** Last message in was {'$gen_cast',reopen_db}
** When Server state == {group_state,undefined,<<"exhaust">>,
{"/var/lib/couchdb",<<"exhaust">>,
{group,
<<26,211,78,5,6,56,56,6,195,233,132,7,12,137,41,86>>,
nil,nil,<<"_design/conflicts">>,<<"javascript">>,
[],
[{view,0,
[<<"list">>],
<<"function(doc) {\n if(doc._conflicts) {\n
emit(doc._conflicts, null);\n }\n}\n">>,
nil,[],[]}],
nil,0,0,nil,nil}},
{group,
<<26,211,78,5,6,56,56,6,195,233,132,7,12,137,41,86>>,
{db,<0.125.0>,<0.128.0>,nil,<<"1307284558924035">>,
<0.118.0>,<0.137.0>,
{db_header,5,407871,0,
{14473890608,{670,14}},
{14473892185,684},
{14473796209,[]},
0,nil,nil,1000},
407871,
{btree,<0.118.0>,
{14473890608,{670,14}},
#Fun<couch_db_updater.7.10053969>,
#Fun<couch_db_updater.8.35220795>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.9.107593676>},
{btree,<0.118.0>,
{14473892185,684},
#Fun<couch_db_updater.10.30996817>,
#Fun<couch_db_updater.11.96515267>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.12.117826253>},
{btree,<0.118.0>,
{14473796209,[]},
#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
407871,<<"exhaust">>,
"/var/lib/couchdb/exhaust.couch",[],[],nil,
{user_ctx,null,[],undefined},
nil,1000,
[before_header,after_header,on_file_open],
false},
<0.2256.0>,<<"_design/conflicts">>,<<"javascript">>,
[],
[{view,0,
[<<"list">>],
<<"function(doc) {\n if(doc._conflicts) {\n
emit(doc._conflicts, null);\n }\n}\n">>,
{btree,<0.2256.0>,
{157,{1,[]}},
#Fun<couch_btree.3.83553141>,
#Fun<couch_btree.4.30790806>,
#Fun<couch_view.less_json_ids.2>,
#Fun<couch_view_group.10.132983779>},
[],[]}],
{btree,<0.2256.0>,
{51,[]},
#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
407871,0,nil,nil},
nil,nil,false,[],<0.2261.0>,<0.511.0>}
** Reason for termination ==
** {timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}}
[Sun, 05 Jun 2011 14:47:29 GMT] [error] [<0.507.0>] {error_report,<0.31.0>,
{<0.507.0>,crash_report,
[[{initial_call,{couch_view_group,init,['Argument__1']}},
{pid,<0.507.0>},
{registered_name,[]},
{error_info,
{exit,
{timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}},
[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
{ancestors,
[couch_view,couch_secondary_services,couch_server_sup,<0.32.0>]},
{messages,[]},
{links,[<0.511.0>,<0.2256.0>,<0.101.0>]},
{dictionary,[]},
{trap_exit,true},
{status,running},
{heap_size,987},
{stack_size,24},
{reductions,1902}],
[{neighbour,
[{pid,<0.511.0>},
{registered_name,[]},
{initial_call,{couch_event_sup,init,['Argument__1']}},
{current_function,{gen_server,loop,6}},
{ancestors,
[<0.507.0>,couch_view,couch_secondary_services,
couch_server_sup,<0.32.0>]},
{messages,[{'$gen_cast',stop}]},
{links,[<0.507.0>,<0.85.0>]},
{dictionary,[]},
{trap_exit,false},
{status,runnable},
{heap_size,233},
{stack_size,9},
{reductions,32}]}]]}}
[Sun, 05 Jun 2011 14:47:35 GMT] [error] [<0.2256.0>] ** Generic server
<0.2256.0> terminating
** Last message in was {'EXIT',<0.507.0>,
{timeout,
{gen_server,call,
[<0.137.0>,{drop,<0.507.0>}]}}}
** When Server state == {file,{file_descriptor,prim_file,{#Port<0.3297>,22}},
0,4185}
** Reason for termination ==
** {timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}}
[Sun, 05 Jun 2011 14:47:35 GMT] [error] [<0.2256.0>] {error_report,<0.31.0>,
{<0.2256.0>,crash_report,
[[{initial_call,{couch_file,init,['Argument__1']}},
{pid,<0.2256.0>},
{registered_name,[]},
{error_info,
{exit,
{timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}},
[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
{ancestors,
[<0.507.0>,couch_view,couch_secondary_services,couch_server_sup,
<0.32.0>]},
{messages,[]},
{links,[#Port<0.3297>,<0.2261.0>]},
{dictionary,[]},
{trap_exit,true},
{status,running},
{heap_size,377},
{stack_size,24},
{reductions,2223}],
[{neighbour,
[{pid,<0.2261.0>},
{registered_name,[]},
{initial_call,{couch_ref_counter,init,['Argument__1']}},
{current_function,{gen_server,loop,6}},
{ancestors,
[<0.507.0>,couch_view,couch_secondary_services,
couch_server_sup,<0.32.0>]},
{messages,
[{'DOWN',#Ref<0.0.0.44077>,process,<0.507.0>,
{timeout,
{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}}}]},
{links,[<0.2256.0>]},
{dictionary,[]},
{trap_exit,false},
{status,runnable},
{heap_size,377},
{stack_size,9},
{reductions,114}]}]]}}
========================
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
