Too many open files

[Jonathan Johnson]

Our couch setup has around 100 databases with a significant number of
views in each database. Every once in a while, couch takes a dive. I
happened to be around this time, and saw this in the logs:


[Wed, 01 Dec 2010 18:09:19 GMT] [error] [<0.102.0>] {error_report,<0.31.0>,
    {<0.102.0>,std_error,
     {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}}}

[Wed, 01 Dec 2010 18:09:19 GMT] [error] [<0.10711.1125>] {error_report,<0.31.0>,
              {<0.10711.1125>,std_error,
               [{application,mochiweb},
                "Accept failed error","{error,emfile}"]}}

[Wed, 01 Dec 2010 18:09:19 GMT] [error] [<0.10711.1125>] {error_report,<0.31.0>,
    {<0.10711.1125>,crash_report,
     [[{initial_call,{mochiweb_socket_server,acceptor_loop,['Argument__1']}},
       {pid,<0.10711.1125>},
       {registered_name,[]},
       {error_info,
           {exit,
               {error,accept_failed},
               [{mochiweb_socket_server,acceptor_loop,1},
                {proc_lib,init_p_do_apply,3}]}},
       {ancestors,
           [couch_httpd,couch_secondary_services,couch_server_sup,<0.32.0>]},
       {messages,[]},
       {links,[<0.102.0>]},
       {dictionary,[]},
       {trap_exit,false},
       {status,running},
       {heap_size,233},
       {stack_size,24},
       {reductions,202}],
      []]}}

[Wed, 01 Dec 2010 18:09:19 GMT] [error] [<0.102.0>] {error_report,<0.31.0>,
    {<0.102.0>,std_error,
     {mochiweb_socket_server,225,{acceptor_error,{error,accept_failed}}}}}

I had run into an open files limit before, and had adjusted a few
settings. Here are some of the config values I think are relevant:

max_dbs_open = 100
max_connections = 2048

>From /etc/security/limits.conf
couchdb         hard    nofile  4096
couchdb         soft    nofile  4096

The installed version is 1.0.1.

I'm not sure how to debug this issue further. It only happens after
several days of usage, and once it happens, I can't even ask for the
stats page to see what the current numbers are :)

Thanks in advance for any help!
-Jon