Mmmh, so the real problem just seems to be, that the external handling
of POST/PUT calls does not handle binary data seperately from String
data. So one has to base64encode those attachments. I'm not really
good enough at erlang coding (yet) to move the code from the usual
couchdb handling (for multipart/form-data) to external handlers, but
that would be really good.
For now I stick with base64 encoding of binary data, which works ok,
since the produced json object sent to my external ruby script does
not contain any offending "chars".
Greetings,
Lennart
On Nov 11, 2009, at 8:29 AM, Lennart Melzer wrote:
Hi Paul,
I double checked the response of my external ruby script.
I reduced the code to the bare minimum (nearly representing what the
python example from the wiki looks like)
#!/usr/bin/env ruby
require 'rubygems'
require 'json'
def request
begin
while line = STDIN.readline
yield JSON.parse(line)
end
rescue EOFError
return
end
end
def respond code = 200, data = {}, headers = {}
output = "#{{:code => code,:body => data, :headers =>
headers}.to_json}\n"
STDOUT.write output
STDOUT.flush
end
request do |line|
respond 200, "random"
end
When I do a GET request to the external handler I get "random" back,
when I do a POST with a simple form I get "random" back, but when
POSTing an image (by curl) as multipart/form-data it breaks with the
aforementioned error returning nothing. So since the return value
seems to work with other types of request, it can't be my external
handler.
Sorry for bothering, but I really want to get this working. Been
reading through the code for some time now, but I don't understand
why couchdb is even calling send_json in the first place.
Greetings,
Lennart
On Nov 11, 2009, at 12:11 AM, Paul Davis wrote:
On Tue, Nov 10, 2009 at 5:56 PM, Lennart Melzer <[email protected]>
wrote:
Hi Paul,
I'm not trying to return arbitrary binary data,I really just try
to post an
attached file to an external handler. Right now the handler
returns a static
response which works when posting form data (without binary data
in it) and
with get request. It doesn't work when posting a binary file. I
don't think
it's a problem with the handler itself.
I don't know why the couchdb is trying to encode the binary post
request as
json.
Greetings,
Lennart
On Nov 10, 2009, at 11:26 PM, Paul Davis wrote:
On Tue, Nov 10, 2009 at 5:22 PM, Lennart Melzer <l.mel...@tu-
bs.de> wrote:
I just looked through the error logs and the code a bit more. I
think
this
passage describes best what seems to happen (though I don't know
why)
So the stracktrace shows that the mochiweb passes the request to
couch_httpd:handle_request where it seems to try to call send_json
(whysoever). It all fails when it tries to encode the binary
data as a
json
string. This all seems to happen before couch_httpd_external.erl
stuff
gets
called, which might be able to handle binary data correctly.
Here's the log snippet:
crasher:
initial call: mochiweb_socket_server:acceptor_loop/1
pid: <0.1059.0>
registered_name: []
exception exit: {ucs,{bad_utf8_character_code}}
in function xmerl_ucs:from_utf8/1
in call from mochijson2:json_encode_string/2
in call from mochijson2:'-json_encode_proplist/2-fun-0-'/3
in call from lists:foldl/3
in call from mochijson2:json_encode_proplist/2
in call from couch_httpd:send_json/4
in call from couch_httpd:handle_request/5
in call from mochiweb_http:headers/5
Anyone out that knows more about the internals of couchdb than I
do? I
would
be grateful for any advice (gotta learn some erlang programming
soonish).
Greetings,
Lennart
On Nov 10, 2009, at 12:13 AM, Paul Davis wrote:
On Mon, Nov 9, 2009 at 12:15 PM, Lennart Melzer <[email protected]
>
wrote:
I am running Revision 832477 of the couchdb trunk
On Nov 9, 2009, at 6:09 PM, Paul Davis wrote:
On Mon, Nov 9, 2009 at 11:54 AM, Lennart Melzer <[email protected]
>
wrote:
Right now, posting or putting binary data to an external
handler
fails,
since the data is parsed as a UTF-8 String and not treated
as binary.
The
external handler never gets to do its work, so this might be
an issue
with
the handling of data sent to an external handler. Am I
misusing the
external-handler interface?
Error snippet:
[error] [<0.95.0>] {error_report,<0.30.0>,
{<0.95.0>,std_error,
{mochiweb_socket_server,235,
{child_error,{ucs,{bad_utf8_character_code}}}}}}
Greetings,
Lennart
Lennart,
What version of CouchDB are you on? I seem to remember a
patch for
this going into trunk at some point. Not sure if that was pre/
post
0.10 though.
Paul Davis
Lennart,
Hmm, looking through the svn log I'm not seeing a commit that
mentions
body or posts. I swear I remember something like that coming
through
at one point. Benoit, didn't you have something to do with that?
Paul Davis
Lennart,
That stack trace suggests that you're trying to return a JSON
response
that is not valid JSON. If your external is trying to return
arbitrary
binary data in the _external response object you'll need to make
sure
and use the body member.
Paul Davis
Lennart,
What are you returning from your _external handler? You should be
returning something like:
{
"code": 200,
"body": "arbitrary string of data and some stuff",
"headers": {"Content-Type": "application/my-stuff"}
}
I'd have to remind myself on the exact rules, but returning {"code":
200, "json": "stuff"} will send "stuff" through the JSON serializer,
which means it has to conform to unicode escaping rules that
mochijson2 exposes via xmerl:ucs16_to_char or whatever.
Perhaps you can log the response you're sending and paste a copy of
that?
HTH,
Paul Davis
