[
https://issues.apache.org/jira/browse/COUCHDB-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12841107#action_12841107
]
Jan Lehnardt commented on COUCHDB-679:
--------------------------------------
I'm trying this patch and it gets to 18/26 before keeping CouchDB using 100%
CPU for minutes until I ctrl-c make check.
> Pull replication no longer works when a source doc has a large attachment
> -------------------------------------------------------------------------
>
> Key: COUCHDB-679
> URL: https://issues.apache.org/jira/browse/COUCHDB-679
> Project: CouchDB
> Issue Type: Bug
> Components: Replication
> Environment: trunk and 0.11
> Reporter: Filipe Manana
> Priority: Blocker
> Fix For: 0.11
>
> Attachments: CouchDB-679-etap_test-trunk.patch
>
>
> After rev r916868 (fix for CouchDB-597), doing a pull replication of a doc
> that has a large attachment no longer works.
> The problem is in couch_rep_att.erl:
> convert_stub(#att{data=stub, name=Name} = Attachment,
> {#http_db{} = Db, Id, Rev}) ->
> {Pos, [RevId|_]} = Rev,
> Request = Db#http_db{
> resource = lists:flatten([couch_util:url_encode(Id), "/",
> couch_util:url_encode(Name)]),
> qs = [{rev, couch_doc:rev_to_str({Pos,RevId})}]
> },
> Ref = make_ref(),
> RcvFun = fun() ->
> Bin = attachment_receiver(Ref, Request),
> cleanup(),
> Bin
> end,
> Attachment#att{data=RcvFun}.
> The cleanup/0 function can not be called there, since when the attachment is
> received in multiple chunks, after receiving the first chunk the subsequent
> ones are silently discarded by cleanup/0:
> cleanup() ->
> receive
> {ibrowse_async_response, _, _} ->
> %% TODO maybe log, didn't expect to have data here
> cleanup();
> {ibrowse_async_response_end, _} ->
> cleanup();
> {ibrowse_async_headers, _, _, _} ->
> cleanup()
> after 0 ->
> erase(),
> ok
> end.
> If you look into couch_db.erl, you'll see that the attachment receiver
> function maybe called multiple times:
> flush_att(Fd, #att{data=Fun,att_len=AttLen}=Att) when is_function(Fun) ->
> with_stream(Fd, Att, fun(OutputStream) ->
> write_streamed_attachment(OutputStream, Fun, AttLen)
> end).
> write_streamed_attachment(_Stream, _F, 0) ->
> ok;
> write_streamed_attachment(Stream, F, LenLeft) ->
> Bin = F(),
> ok = couch_stream:write(Stream, Bin),
> write_streamed_attachment(Stream, F, LenLeft - size(Bin)).
> This is serious :(
> However, removing that call to cleanup/0 may cause the return of CouchDB-597,
> therefore I don't supply a patch here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
