Re: CouchDB 2.3.1 - Really Big Document Breaks View
You also need to raise the os_process_timeout
Cheers
Jan
—
> On 9. Apr 2020, at 22:29, Mantell, Christopher Arthur
> wrote:
>
> Hi,
>
> One document in our database is really big (1.2 million lines) and it causes
> an OS_process_error when we access it. I've tried increasing the
> os_process_timeout to 12 but it didn't fix it. I don't know if this
> should be increased further. Deleting the document causes the view to work
> again.
>
> Unfortunately there is no ideal way around not using this document or
> splitting it up into smaller documents. Is there anything I can do to
> Couch's configurations so this document won't break the view?
>
> Here is the output in the couch.log:
>
> [error] 2020-04-09T17:29:05.046811Z couchdb@127.0.0.1 <0.2218.0> 4e36c78fcd
> rexi_server: from: couchdb@127.0.0.1(<0.745.0>) mfa: fabric_rpc:map_view/5
> exit:timeout [{rexi,init_stream,1,[{file,"src/rexi.e\
> rl"},{line,265}]},{rexi,stream2,3,[{file,"src/rexi.erl"},{line,205}]},{fabric_rpc,view_cb,2,[{file,"src/fabric_rpc.erl"},{line,462}]},{couch_mrview,map_fold,3,[{file,"src/couch_mrview.erl"},{line,526}]},\
> {couch_mrview_util,fold_fun,4,[{file,"src/couch_mrview_util.erl"},{line,437}]},{couch_btree,stream_kv_node2,8,[{file,"src/couch_btree.erl"},{line,848}]},{couch_btree,stream_kp_node,7,[{file,"src/couch_bt\
> ree.erl"},{line,775}]},{couch_btree,fold,4,[{file,"src/couch_btree.erl"},{line,219}]}]
>
> Any help would be greatly appreciated!
>
> Thank you,
>
>
> Chris Mantell
>
> Programmer
>
> Athinoula A Martinos Center for Biomedical Imaging
> Massachusetts General Hospital
> Building 149 - South Central
> Charlestown, MA 02129
> cmant...@mgh.harvard.edu
>
>
> The information in this e-mail is intended only for the person to whom it is
> addressed. If you believe this e-mail was sent to you in error and the e-mail
> contains patient information, please contact the Partners Compliance HelpLine
> at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
Re: CouchDB 2.3.1 - Really Big Document Breaks View
We had a similar problem we fixed by having couchjs allocate more RAM.
We used the Environment Variable to configure the Query Servers like
this:
COUCHDB_QUERY_SERVER_JAVASCRIPT="/usr/local/libexec/couchdb2/bin/couchjs
-S 536870912 /usr/local/libexec/couchdb2/share/server/main.js"
Obviously feel free to tune the memory size (-S) and the path.
Consider that Couch can run many couchjs at the same time, so plan the
-S parameter accordingly.
---
Andrea Brancatelli
On 2020-04-09 22:29, Mantell, Christopher Arthur wrote:
> Hi,
>
> One document in our database is really big (1.2 million lines) and it causes
> an OS_process_error when we access it. I've tried increasing the
> os_process_timeout to 12 but it didn't fix it. I don't know if this
> should be increased further. Deleting the document causes the view to work
> again.
>
> Unfortunately there is no ideal way around not using this document or
> splitting it up into smaller documents. Is there anything I can do to
> Couch's configurations so this document won't break the view?
>
> Here is the output in the couch.log:
>
> [error] 2020-04-09T17:29:05.046811Z couchdb@127.0.0.1 <0.2218.0> 4e36c78fcd
> rexi_server: from: couchdb@127.0.0.1(<0.745.0>) mfa: fabric_rpc:map_view/5
> exit:timeout [{rexi,init_stream,1,[{file,"src/rexi.e\
> rl"},{line,265}]},{rexi,stream2,3,[{file,"src/rexi.erl"},{line,205}]},{fabric_rpc,view_cb,2,[{file,"src/fabric_rpc.erl"},{line,462}]},{couch_mrview,map_fold,3,[{file,"src/couch_mrview.erl"},{line,526}]},\
> {couch_mrview_util,fold_fun,4,[{file,"src/couch_mrview_util.erl"},{line,437}]},{couch_btree,stream_kv_node2,8,[{file,"src/couch_btree.erl"},{line,848}]},{couch_btree,stream_kp_node,7,[{file,"src/couch_bt\
> ree.erl"},{line,775}]},{couch_btree,fold,4,[{file,"src/couch_btree.erl"},{line,219}]}]
>
> Any help would be greatly appreciated!
>
> Thank you,
>
> Chris Mantell
>
> Programmer
>
> Athinoula A Martinos Center for Biomedical Imaging
> Massachusetts General Hospital
> Building 149 - South Central
> Charlestown, MA 02129
> cmant...@mgh.harvard.edu
>
> The information in this e-mail is intended only for the person to whom it is
> addressed. If you believe this e-mail was sent to you in error and the e-mail
> contains patient information, please contact the Partners Compliance HelpLine
> at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
CouchDB 2.3.1 - Really Big Document Breaks View
Hi,
One document in our database is really big (1.2 million lines) and it causes an
OS_process_error when we access it. I've tried increasing the
os_process_timeout to 12 but it didn't fix it. I don't know if this should
be increased further. Deleting the document causes the view to work again.
Unfortunately there is no ideal way around not using this document or splitting
it up into smaller documents. Is there anything I can do to Couch's
configurations so this document won't break the view?
Here is the output in the couch.log:
[error] 2020-04-09T17:29:05.046811Z couchdb@127.0.0.1 <0.2218.0> 4e36c78fcd
rexi_server: from: couchdb@127.0.0.1(<0.745.0>) mfa: fabric_rpc:map_view/5
exit:timeout [{rexi,init_stream,1,[{file,"src/rexi.e\
rl"},{line,265}]},{rexi,stream2,3,[{file,"src/rexi.erl"},{line,205}]},{fabric_rpc,view_cb,2,[{file,"src/fabric_rpc.erl"},{line,462}]},{couch_mrview,map_fold,3,[{file,"src/couch_mrview.erl"},{line,526}]},\
{couch_mrview_util,fold_fun,4,[{file,"src/couch_mrview_util.erl"},{line,437}]},{couch_btree,stream_kv_node2,8,[{file,"src/couch_btree.erl"},{line,848}]},{couch_btree,stream_kp_node,7,[{file,"src/couch_bt\
ree.erl"},{line,775}]},{couch_btree,fold,4,[{file,"src/couch_btree.erl"},{line,219}]}]
Any help would be greatly appreciated!
Thank you,
Chris Mantell
Programmer
Athinoula A Martinos Center for Biomedical Imaging
Massachusetts General Hospital
Building 149 - South Central
Charlestown, MA 02129
cmant...@mgh.harvard.edu
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
