Re: transient errors with couch DB.

[Valentin Kuznetsov]

Chris,




Valentin

On Jul 3, 2009, at ,Jul 3, 2:46 PM, Chris Anderson wrote:

thank you for suggestions. The problem is that we tried out all of those and encounter other problems. If I inject 64K rows at once, then memory consumption as well as CPU got high. Moreover the view after that takes literally more then 10 minutes to compute. We're working in environment where 1000 simultaneous requests are possible. Our preliminary tests shown that view computation grows significantly with number of inserted docs. So what I present we found optimal in our situation. Regarding sockets. It well may be and I'll be willing to explore this. But the code example I posted did sequential insertion of docs. Basically open HTTP connection, insert docs, close it.

On Thu, Jul 2, 2009 at 7:35 AM, Valentin Kuznetsov<vkuz...@gmail.com> wrote:

Hi,

large documents, e.g. 64K, which split into 1K chunks (rows) for bulk

to force view to index data.

I'm developing application based on couch db and time to time experience different error messages and crashes with couch, version 0.9.0. We have a injection. Each time I inject new chunk I also invoke 1 GET request w/ 1 row

You should not be querying the view on every insert, once every few
thousand docs should be fine...

I'd maybe go ahead and not break up the 64Kb docs into smaller piece,
Couch should be able to handle docs of that size just fine.

A quick google reveals that einval has got something to do with
opening sockets. My guess is that you are opening too many at once for
your OS. You may be able to do some linux config to raise the limit,
but I'd start by simplifying your insert loop.

So here are error messages:

[error] [<0.12304.2>] Uncaught error in HTTP request: {exit,normal}

[error] [<0.30955.5>] {error_report,<0.23.0>,
   {<0.30955.5>,crash_report,

      {pid,<0.30955.5>},
      {registered_name,[]},
      {error_info,
          {error,
              {case_clause,{error,einval}},
              [{mochiweb_request,get,2},
               {couch_httpd,handle_request,4},
               {mochiweb_http,headers,5},
               {proc_lib,init_p_do_apply,3}]}},
      {ancestors,

      {messages,[]},
      {links,[<0.53.0>,#Port<0.1735>]},
      {dictionary,[{mochiweb_request_qs,[{"limit","1"}]}]},
      {trap_exit,false},
      {status,running},
      {heap_size,6765},
      {stack_size,24},
      {reductions,3888}],
     []]}}
[error] [<0.53.0>] {error_report,<0.23.0>,
   {<0.53.0>,std_error,



=ERROR REPORT==== 2-Jul-2009::10:21:33 ===


So, I made a stand-alone program and to my surprise the errors are

those).


which we use. To run a test untar archive and do the following

export PYTHONPATH=$PWD
python couch_test.py





Thank you,
Valentin

[[{initial_call,{mochiweb_socket_server,acceptor_loop, ['Argument__1']}}, [couch_httpd,couch_secondary_services,couch_server_sup,<0.1.0>]}, {mochiweb_socket_server,235,{child_error,{case_clause, {error,einval}}}}}} {mochiweb_socket_server,235,{child_error,{case_clause, {error,einval}}}} transient. I re-run my stand-alone program with the same doc several times and I can either inject all data just fine or get errors shown above in no particular order (I mean I can see 1 or more errors and none or either of I would appreciate any help with this. The program is attached in a form of tar archive. I added data file we usually deal with and two python files

--
Chris Anderson
http://jchrisa.net
http://couch.io

smime.p7s
Description: S/MIME cryptographic signature