Thanks Koji On Tue, Feb 28, 2017 at 8:44 AM, Koji Kawamura <[email protected]> wrote:
> Mohammed, > > NiFi REST API is for managing NiFi itself and data flow running on it, > as Andy described earlier, it's not for data transfer. It does have > endpoints to receive POST requests but those are for adding Processors > or Templates ... etc. Please refer NiFi REST API docs for details. > https://nifi.apache.org/docs/nifi-docs/rest-api/index.html > > Thanks, > Koji > > On Tue, Feb 28, 2017 at 9:25 PM, mohammed shambakey > <[email protected]> wrote: > > Thank you all for your help > > > > I think Koji's thread at JIRA will be the best solution when it is > > implemented > > > > Just a quick question, Andy mentioned REST APIs for NIFI. I didn't find > much > > examples about using NIFI REST APIs. Is it possible to POST the WEB UI > > (including all inputs from multipart form and uploaded files in a POST > > request to a NIFI input port of a remote NIFI instance? because, I think, > > this will avoid using "handlehttprequest" processor and allow users to > > communicate with the remote NIFI instance both from WEB UI and/or NIFI > > instance (e.g., site-to-site). > > > > Regards > > > > On Mon, Feb 27, 2017 at 6:54 PM, Koji Kawamura <[email protected]> > > wrote: > >> > >> If the multipart contains only text data, it might be possible to use > >> those processors Andy suggested. However, if we support multipart > >> natively, it can be done much easier and safer. I created a JIRA for > >> this improvement before, but haven't been able to find a time to > >> implement it myself. > >> If anyone interested in tackling this enhancement, it'd be greatly > >> appreciated. > >> https://issues.apache.org/jira/browse/NIFI-3469 > >> > >> Thanks, > >> Koji > >> > >> On Tue, Feb 28, 2017 at 4:59 AM, Andy LoPresto <[email protected]> > >> wrote: > >> > Absolutely. Use some combination of SplitText, SplitContent, > SplitJSON, > >> > and > >> > ExtractText to split the content into multiple flowfiles on regular > >> > expressions, new lines, and/or byte sequences and then RouteOnContent > to > >> > send the flowfiles which reference a file to a processor to > >> > retrieve/parse > >> > the file contents. > >> > > >> > Andy LoPresto > >> > [email protected] > >> > [email protected] > >> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> > > >> > On Feb 27, 2017, at 2:08 PM, mohammed shambakey <[email protected] > > > >> > wrote: > >> > > >> > This is an example of the output file from "handlehttprequest": > >> > > >> > " > >> > -----------------------------831805444208556434014280960^M > >> > Content-Disposition: form-data; name="userid"^M > >> > ^M > >> > jpl_user^M > >> > -----------------------------831805444208556434014280960^M > >> > Content-Disposition: form-data; name="scripts"; filename="a.py"^M > >> > Content-Type: text/x-python^M > >> > ^M > >> > ^M > >> > -----------------------------831805444208556434014280960^M > >> > Content-Disposition: form-data; name="scripts"; filename="b.py"^M > >> > Content-Type: text/x-python^M > >> > ^M > >> > ^M > >> > -----------------------------831805444208556434014280960^M > >> > Content-Disposition: form-data; name="scripts"; filename="conf.json"^M > >> > Content-Type: application/json^M > >> > ^M > >> > { > >> > "userid":"jpl_user_1", > >> > "scripts":["a.py","b.py"] > >> > } > >> > ^M > >> > -----------------------------831805444208556434014280960--^M > >> > " > >> > > >> > Can I extract different components of this file without script at > server > >> > side? > >> > > >> > Regards > >> > > >> > On Mon, Feb 27, 2017 at 10:52 AM, Andy LoPresto <[email protected] > > > >> > wrote: > >> >> > >> >> Mohammed, > >> >> > >> >> So from what I understand you have a separate web application which > >> >> accepts a variety of data (a combination of form fields and file > >> >> uploads) > >> >> and submits that (I’m assuming via POST) to NiFi. I’m not sure what > >> >> problem > >> >> you had extracting these “components” from the output flowfile from > the > >> >> HandleHTTPRequest processor. Can you change your web interface to > send > >> >> separate POST requests to NiFi and logically separate these > components > >> >> from > >> >> the beginning? Can you modify the format of the request (i.e. make it > >> >> JSON, > >> >> etc.) so it is properly structured for NiFi to parse and operate on > >> >> this > >> >> data? > >> >> > >> >> NiFi treats the content of the flowfile the same once it is in the > >> >> system > >> >> regardless of the source processor, so you should be able to split, > >> >> extract, > >> >> or manipulate the contents of a single flowfile containing multiple > >> >> form > >> >> elements and file bodies. > >> >> > >> >> If you are referencing NiFi’s Input Port components as a destination > >> >> for > >> >> an arbitrary HTTP request, this will not work. The input ports are > >> >> designed > >> >> to work with NiFi’s Site to Site protocol (either over raw sockets or > >> >> HTTP(S)). > >> >> > >> >> > >> >> Andy LoPresto > >> >> [email protected] > >> >> [email protected] > >> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> >> > >> >> On Feb 27, 2017, at 11:09 AM, mohammed shambakey < > [email protected]> > >> >> wrote: > >> >> > >> >> Thanks Andy > >> >> > >> >> I have a user web interface that send some form information (e.g., > user > >> >> id), as well as upload multiple files, to a remote NIFI instance. > When > >> >> using > >> >> "handlhttprequest" processor, it results in a single file that > contains > >> >> "form inputs" and bodies of uploaded files. > >> >> > >> >> I want to extract individual components of this file (i.e., download > >> >> all > >> >> uploaded files and extract other form information (e.g., userid) in a > >> >> JSON > >> >> or other text file. I had a similar problem before and I learnt that > >> >> "httphandlerequest" has a problem to extract individual compoents > from, > >> >> so I > >> >> have to write my own scripts to extract these components (form inputs > >> >> and > >> >> uploaded files). > >> >> > >> >> As NIFI input ports does not face this problem of > "handlehttprequest", > >> >> I > >> >> though if I can send form inputs and upload files directly to "NIFI > >> >> input > >> >> port" at remote site, then I don't have to write my own scripts to > >> >> extract > >> >> individual components. > >> >> > >> >> Regards > >> >> > >> >> On Mon, Feb 27, 2017 at 10:02 AM, Andy LoPresto < > [email protected]> > >> >> wrote: > >> >>> > >> >>> Mohammed, > >> >>> > >> >>> I’m not sure I understand your question. NiFi has a REST API which > >> >>> supports the UI and allows scripts/other applications to make > requests > >> >>> of > >> >>> the platform. This runs on the input port specified in > >> >>> nifi.properties. > >> >>> > >> >>> If you want to submit form data and uploaded files as data to be > >> >>> ingested > >> >>> by NiFi, HandleHTTPRequest or (less likely) ListenHTTP are the > >> >>> processors > >> >>> you would want to use. > >> >>> > >> >>> If you can clarify your use case, we should be able to offer more > >> >>> assistance. > >> >>> > >> >>> Andy LoPresto > >> >>> [email protected] > >> >>> [email protected] > >> >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> >>> > >> >>> On Feb 27, 2017, at 9:15 AM, mohammed shambakey < > [email protected]> > >> >>> wrote: > >> >>> > >> >>> Hi > >> >>> > >> >>> Is it possible to send http request (including form data and > uploaded > >> >>> files) to NIFI input port instead of "HandleHTTPRequest" processor? > >> >>> > >> >>> Regards > >> >>> > >> >>> -- > >> >>> Mohammed > >> >>> > >> >>> > >> >> > >> >> > >> >> > >> >> -- > >> >> Mohammed > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Mohammed > >> > > >> > > > > > > > > > > > -- > > Mohammed > -- Mohammed
