Hi all,
This is a last-ditch attempt to resolve the issue of POSTing
multipart/form-data using InvokeHTTP. Our alternatives would be to either not
using NiFi for this use-case - which would be a pity - or writing our own
processor that can properly submit forms.
Our flow, NiFi 1.9.2:
Issues found so far:
[HandleHTTPRequest] - [ExtractText] - [ReplaceText] - [InvokeHTTP] -
[HandleHTTPResponse]
ExtractText pulls the flowfile data into an attribute;
ReplaceText builds a message as per RFC2388 (tried several variations of this,
including with/without newlines in the expression editor):
--------------------------555791708927339309556823${literal(‘\r\n’)}
Content-Disposition: form-data; name=“data_src”${literal(‘\r\n’)}
${literal(‘\r\n’)}
TST${literal(‘\r\n’)}
--------------------------555791708927339309556823${literal(‘\r\n’)}
Content-Disposition: form-data; name=“cmd”${literal(‘\r\n’)}
${literal(‘\r\n’)}
sendResultDetails${literal(‘\r\n’)}
--------------------------555791708927339309556823${literal(‘\r\n’)}
Content-Disposition: form-data; name=“cmd_data”;
filename=“${filename}“${literal(‘\r\n’)}
Content-Type: text/plain${literal(‘\r\n’)}${literal(‘\r\n’)}
${cmd_data}
${literal(‘\r\n’)}
--------------------------555791708927339309556823--${literal(‘\r\n’)}
InvokeHTTP POSTs message to httpbin.org for testing:
Set URL, etc.
Set Content-Type to: multipart/form-data;
boundary=--------------------------555791708927339309556823
(We also tried setting Content-Type to text/plain and application/octet-stream,
and adding Content-Type: multipart/form-data;
boundary=--------------------------555791708927339309556823 to the top of the
body, none of these options worked)
Add "Accept: */*" Header
Where we use the Content-Type: multipart/form-data configuration, httpbin.org
doesn't see any data at all. Where we use a different Content-Type, httpdin.org
just parses the complete message body as a standard message body.
If we build the exact same message with, for example, Postman, we receive valid
results. Sending both NiFi and Postman POSTs yields identical output when
captured on a listening netcat port.
The only answer I have managed to find on the question of "How to POST
form-data with InvokeHTTP is "build your own message body". Most of these
questions are met with silence. It would be extremely helpful if anyone could
comment on this actually being possible with InvokeHTTP.
Any feedback or assistance would be appreciated.
On Fri, 26 Apr 2019, at 13:27, Martijn Dekkers wrote:
> Hi,
>
> I am working to implement a flow where we need to POST some data to another
> system using multipart/form-data. This system requires that one of the form
> fields acts as an attached file. Using curl, this works fine:
>
> curl -F <field1>=<data1> -F <field2>=<data2> -F <field3>=@[filename] <URL>
>
> I created a flowfile that mimics what curl sends, including the boundaries
> etc. Using InvokeHTTP POST I receive a web-server error from the receiving
> end that mentions a missing parameter.
>
> Running with curl --trace-ascii or POSTing to a running netcat with both curl
> and nifi shows exactly the same results, with the exception of the actual
> boundary marker (which is always different) and some minor header stuff (UA,
> etc.) The payload is simple text.
>
> I understand that there is some difference between sending the data and
> uploading a file, but am unable to figure out what this is. for example,
> sending the same data with curl using -F<field3>=<[filename] (instead of
> @[filename]) also results in an error.
>
> I have read a few different messages about POSTing multipart-form data where
> files need to be uploaded, and most appear to end up with writing a scripted
> processor or invoking shell. This is particularly undesirable for us given
> the volume of files we must process, and the speed at which we should be
> uploading those files.
>
> Is there any way in which to mimic the uploading/attaching of a file, instead
> of sending the data as text?
>
> Thanks
>
> Martijn
