Likely couchdb is hitting the file handle limit. Be sure you're test suite
is closing unused connections and report back. Couch shouldn't leak handles.
On Sep 12, 2010 9:44 AM, "John" <[email protected]> wrote:
> Hi
>
> I have a consistent problem every time I try to do a very simple load test
> on the same URI. Basically I hit it every few milliseconds and after approx
> 1000 hits I get this error:
>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target: ./lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target: ./lib.beam. Function: get_file.
> Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/kernel-2.13.5/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
>
> The entire stack trace is at the end of this email.
>
> I am running 1.0.0.8 on OSX 10.6.4 on a little macbook air! The error might
> be related to my test client which is an experimental NIO Client Connector
> for an Open Source product. I suspect it's doing something naughty with
> connections and exhausting the filehandles on my baby-mac but in any case it
> would be good to get some clarification on that. Can anyone help?
>
>
> Regards
>
> John
>
>
> Entire Stack Trace:
>
>
> 1> [info] [<0.1242.0>] 127.0.0.1 - - 'GET'
> /uta1/d6d81b828f20b36ee18190b83000233e 200
> 1> [info] [<0.1243.0>] 127.0.0.1 - - 'GET'
> /uta1/d6d81b828f20b36ee18190b83000233e 200
> 1> [info] [<0.1244.0>] 127.0.0.1 - - 'GET'
> /uta1/d6d81b828f20b36ee18190b83000233e 200
> 1> [info] [<0.1245.0>] 127.0.0.1 - - 'GET'
> /uta1/d6d81b828f20b36ee18190b83000233e 200
> 1> [info] [<0.1246.0>] 127.0.0.1 - - 'GET'
> /uta1/d6d81b828f20b36ee18190b83000233e 200
> 1> [info] [<0.1247.0>] 127.0.0.1 - - 'GET'
> /uta1/d6d81b828f20b36ee18190b83000233e 200
> 1> [info] [<0.1248.0>] 127.0.0.1 - - 'GET'
> /uta1/d6d81b828f20b36ee18190b83000233e 200
> 1> [error] [<0.1249.0>] {error_report,<0.35.0>,
> {<0.1249.0>,std_error,
> [{application,mochiweb},
> "Accept failed error","{error,enfile}"]}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> application: mochiweb
> "Accept failed error"
> "{error,enfile}"
> 1> [error] [<0.1249.0>] {error_report,<0.35.0>,
> {<0.1249.0>,crash_report,
>
> [[{initial_call,{mochiweb_socket_server,acceptor_loop,['Argument__1']}},
> {pid,<0.1249.0>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_socket_server,acceptor_loop,1},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
>
> [couch_httpd,couch_secondary_services,couch_server_sup,<0.36.0>]},
> {messages,[]},
> {links,[<0.105.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,199}],
> []]}}
> 1> [error] [<0.105.0>] {error_report,<0.35.0>,
> {<0.105.0>,std_error,
> {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target: ./lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target: ./lib.beam. Function: get_file.
> Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/kernel-2.13.5/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/kernel-2.13.5/ebin/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/stdlib-1.16.5/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/stdlib-1.16.5/ebin/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/mochiweb-r113/ebin/lib.beam. Function:
> get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/mochiweb-r113/ebin/lib.beam. Function:
> get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/ibrowse-1.5.2/ebin/lib.beam. Function:
> get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/ibrowse-1.5.2/ebin/lib.beam. Function:
> get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/etap/ebin/lib.beam. Function: get_file.
> Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/etap/ebin/lib.beam. Function: get_file.
> Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/erlang-oauth/ebin/lib.beam. Function:
> get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/erlang-oauth/ebin/lib.beam. Function:
> get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/couch-1.0.0/ebin/lib.beam. Function:
> get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> couchdb_1.0.0/lib/couchdb/erlang/lib/couch-1.0.0/ebin/lib.beam. Function:
> get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/xmerl-1.2.4/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/xmerl-1.2.4/ebin/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/ssl-3.10.8/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/ssl-3.10.8/ebin/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/sasl-2.1.9/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/sasl-2.1.9/ebin/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/inets-5.3/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/inets-5.3/ebin/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/erl_docgen-0.2/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/erl_docgen-0.2/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/crypto-1.6.4/ebin/lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target:
> /Applications/CouchDBX.app/Contents/Resources/couchdbx-core/erlang_R13B04/lib/erlang/lib/crypto-1.6.4/ebin/lib.beam.
> Function: get_file. Process: code_server.
> 1> [error] [<0.20.0>] {error_report,<0.9.0>,
> {<0.20.0>,std_error,
> "File operation error: system_limit. Target: lib.beam.
> Function: get_file. Process: code_server."}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> File operation error: system_limit. Target: lib.beam. Function: get_file.
> Process: code_server.
> 1> [error] [<0.6.0>] ** gen_event handler sasl_report_tty_h crashed.
> ** Was installed in error_logger
> ** Last event was: {error_report,<0.35.0>,
> {<0.1249.0>,crash_report,
> [[{initial_call,
> {mochiweb_socket_server,acceptor_loop,
> ['Argument__1']}},
> {pid,<0.1249.0>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_socket_server,acceptor_loop,1},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.36.0>]},
> {messages,[]},
> {links,[<0.105.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,199}],
> []]}}
> ** When handler state == error
> ** Reason == {'module could not be loaded',
> [{lib,format_exception,
> [5,exit,
> {error,accept_failed},
> [{mochiweb_socket_server,acceptor_loop,1},
> {proc_lib,init_p_do_apply,3}],
>
> #Fun<proc_lib.0.73600383>,#Fun<proc_lib.1.130404962>]},
> {proc_lib,format_exception,3},
> {proc_lib,format_rep,1},
> {proc_lib,format_rep,1},
> {proc_lib,format_rep,1},
> {proc_lib,format,1},
> {sasl_report,write_report2,5},
> {sasl_report_tty_h,handle_event,2}]}
>
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:01 ===
> ** gen_event handler sasl_report_tty_h crashed.
> ** Was installed in error_logger
> ** Last event was: {error_report,<0.35.0>,
> {<0.1249.0>,crash_report,
> [[{initial_call,
> {mochiweb_socket_server,acceptor_loop,
> ['Argument__1']}},
> {pid,<0.1249.0>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_socket_server,acceptor_loop,1},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.36.0>]},
> {messages,[]},
> {links,[<0.105.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,199}],
> []]}}
> ** When handler state == error
> ** Reason == {'module could not be loaded',
> [{lib,format_exception,
> [5,exit,
> {error,accept_failed},
> [{mochiweb_socket_server,acceptor_loop,1},
> {proc_lib,init_p_do_apply,3}],
>
> #Fun<proc_lib.0.73600383>,#Fun<proc_lib.1.130404962>]},
> {proc_lib,format_exception,3},
> {proc_lib,format_rep,1},
> {proc_lib,format_rep,1},
> {proc_lib,format_rep,1},
> {proc_lib,format,1},
> {sasl_report,write_report2,5},
> {sasl_report_tty_h,handle_event,2}]}
> 1> [error] [<0.1250.0>] {error_report,<0.35.0>,
> {<0.1250.0>,std_error,
> [{application,mochiweb},
> "Accept failed error","{error,enfile}"]}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:03 ===
> application: mochiweb
> "Accept failed error"
> "{error,enfile}"
> 1> [error] [<0.1250.0>] {error_report,<0.35.0>,
> {<0.1250.0>,crash_report,
>
> [[{initial_call,{mochiweb_socket_server,acceptor_loop,['Argument__1']}},
> {pid,<0.1250.0>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_socket_server,acceptor_loop,1},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
>
> [couch_httpd,couch_secondary_services,couch_server_sup,<0.36.0>]},
> {messages,[]},
> {links,[<0.105.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,199}],
> []]}}
> 1> [error] [<0.105.0>] {error_report,<0.35.0>,
> {<0.105.0>,std_error,
> {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:03 ===
> {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}
> 1> [error] [<0.1253.0>] {error_report,<0.35.0>,
> {<0.1253.0>,std_error,
> [{application,mochiweb},
> "Accept failed error","{error,enfile}"]}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:05 ===
> application: mochiweb
> "Accept failed error"
> "{error,enfile}"
> 1> [error] [<0.1253.0>] {error_report,<0.35.0>,
> {<0.1253.0>,crash_report,
>
> [[{initial_call,{mochiweb_socket_server,acceptor_loop,['Argument__1']}},
> {pid,<0.1253.0>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_socket_server,acceptor_loop,1},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
>
> [couch_httpd,couch_secondary_services,couch_server_sup,<0.36.0>]},
> {messages,[]},
> {links,[<0.105.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,199}],
> []]}}
> 1> [error] [<0.105.0>] {error_report,<0.35.0>,
> {<0.105.0>,std_error,
> {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}}}
> 1>
> =ERROR REPORT==== 12-Sep-2010::17:25:05 ===
> {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}
> 1>
