Andy, Thanks. Yes, I know how Ajax works having used it before. It's looking at it happening in the browser developer tools that I've never learned to do. I've been using </> RESTED so that I can set Accept application/json which works perfectly.
Russ On Fri, Jul 8, 2016 at 10:55 PM, Andy LoPresto <[email protected]> wrote: > Russell, > > No worries. The Javascript application that runs the UI is simply invoking > Ajax (Asynchronous Javascript and XML) requests to normal HTTP endpoints > provided by the embedded Jetty server in NiFi. You don’t need to understand > anything about front-end development to monitor these requests using the > Dev Tools. In fact, you could simply type the entire API endpoint into the > browser address bar and receive the XML (or JSON-formatted, if you provide > a header requesting it) response from the server directly. If you are more > comfortable on the command-line or in an IDE, you can use Java, Groovy, > etc. or even curl to invoke the API purely through code. There are also API > tools (no endorsement, but some are listed here [1]) for hitting these APIs > from a GUI application. > > Here is a good starting point for using the Network tab of the developer > tools window [2]. > > Glad to hear you have been able to write some custom processors. Let us > know if you have any additional questions. > > [1] > https://stackoverflow.com/questions/4486658/what-tools-do-you-use-to-test-your-public-rest-api > [2] http://commandlinefanatic.com/cgi-bin/showarticle.cgi?article=art034 > > Andy LoPresto > [email protected] > *[email protected] <[email protected]>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jul 8, 2016, at 7:42 PM, Russell Bateman < > [email protected]> wrote: > > (Make that "not much of a web page developer". I'm a back-end Java guy > mostly, don't much value my UI skills and so have stayed away from the > browser except for straight HTML and CSS.) > > On Fri, Jul 8, 2016 at 8:39 PM, Russell Bateman < > [email protected]> wrote: > > Thanks, Andy and Matt, I had actually tried out the Developer Tools > earlier today as suggested by the doc, but, I'm not much of a web > developer, don't have a lot of familiarity with those tools in Chrome or in > Firefox, and I couldn't see anything going on (in Firefox) that was > helpful. I may have to grow strong that way. > > For now, we're in 0.6.1. I'm hoping that soon after 1.0 makes its > appearance, we can move up, but you point out something we're already > beginning to pay attention to: incompatiblities. Still we're committed to > (and love!) the technology. I've written a dozen custom processors (for > proprietary ETL stuff) already and my colleague has done the same. > > Thanks, again. > > > On Fri, Jul 8, 2016 at 5:11 PM, Andy LoPresto <[email protected]> > wrote: > > Russell, > > To echo Matt’s point, the NiFi UI uses the REST API on every command, and > developer tools is a great way to observe this. > > Another example using the REST API is Andrew Grande’s “nifi-deploy-api” > project on GitHub [1]. It is a groovy script that can automate the > deployment of NiFi instances and does this using the API. If you run it > with the ‘--debug’ flag, you can see the HTTP endpoints it invokes to > accomplish this. > > Finally, the REST API is extremely well documented here [2] (also locally > in the installation). It is extensive, but should be able to answer a > number of the questions you asked. For example, to get processor > statistics, you would invoke: > > GET /controller/process-groups/{process-group-id}/status > > Gets the status for a process group > The status for a process group includes status for all descendent > components. When invoked on the root group with recursive set to true, it > will return the current status of every component in the flow. > > When I invoke that endpoint on a simple cluster I have running with a > single Base64EncodeContent processor named “Token Example” on the canvas, > I get this response: > > > > https://ncm.nifi.apache.org:4567/nifi-api/controller/process-groups/root/status > > <processGroupStatusEntity> > <revision> > <clientId>392b161b-d9cc-442a-9394-8db9cee33ba7</clientId> > </revision> > <processGroupStatus> > <activeThreadCount>0</activeThreadCount> > <id>d17d7a95-6301-46ff-b003-11c58698bd31</id> > <input>0 / 0 bytes</input> > <name>NiFi Flow</name> > <output>0 / 0 bytes</output> > <processorStatus> > <activeThreadCount>0</activeThreadCount> > <groupId>d17d7a95-6301-46ff-b003-11c58698bd31</groupId> > <id>b642a34d-c18c-4bb1-b836-6bbc9d32f31e</id> > <input>0 / 0 bytes</input> > <name>*Token Example*</name> > <output>0 / 0 bytes</output> > <read>0 bytes</read> > <runStatus>Invalid</runStatus> > <tasks>0</tasks> > <tasksDuration>00:00:00.000</tasksDuration> > <type>*Base64EncodeContent*</type> > <written>0 bytes</written> > </processorStatus> > <queued>0 / 0 bytes</queued> > <queuedCount>0</queuedCount> > <queuedSize>0 bytes</queuedSize> > <read>0 bytes</read> > <received>0 / 0 bytes</received> > <sent>0 / 0 bytes</sent> > <statsLastRefreshed>16:09:46 PDT</statsLastRefreshed> > <transferred>0 / 0 bytes</transferred> > <written>0 bytes</written> > </processGroupStatus> > </processGroupStatusEntity> > > [1] https://github.com/aperepel/nifi-api-deploy > [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html > > Andy LoPresto > [email protected] > *[email protected] <[email protected]>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jul 8, 2016, at 3:18 PM, Matt Gilman <[email protected]> wrote: > > Russell, > > I can provide a more detailed response when I'm back in front of my > computer but thought I'd offer this as a quick suggestion. > > All flows use process groups. The blank canvas when you load your nifi > instance is the root level process group. If you don't know the actual ID, > you can use the alias 'root' as the path parameter for the process group > ID. > > Please check out your browser's Dev Tools to see these API's in action. > The UI uses these API's exclusively. > > Also the API's have been completely refactored in the upcoming 1.0.0 > release to align with the authorization policies of each resource. So the > answers to your questions will depend on which version your running. > > Matt > > Sent from my iPhone > > On Jul 8, 2016, at 5:35 PM, Russell Bateman < > [email protected]> wrote: > > I'm trying to figure out how to use the ReST (nifi-api) interface to > accomplish a number of things. First, I've played successfully with it > doing simple things like getting configuration, a list of existing > processors, and the like. I've even discovered that in > controller/history/processors/{processorId} I can use the processor name > for {processorId} instead of an intangible or impossible number. > > What puzzles me is that I need a {process-group-id} to do so many things. > I'm not using a process group, my flow is very simple for now. Maybe, > think > I, I have one process group and I just need its id, but how do I get that? > I see no list API for it. > > The long list of things I'm hoping to do down the road from a UI is to be > able to see things like: > > 1. How many times a processor processed anything? > 2. How many flowfiles were processed? > 3. How many flowfiles were produced? > 4. Can I get this information per processor? > 5. How can I tell different instances of the same processor apart? > 6. Why can't I see the name I gave a processor in configuration? (For > example, I named an instance of GetFile to "Get PDF files from test > fodder". > 7. How can I get a list of items on the NiFi canvas like processors and > relationships. > 8. How many times processors or a processor failed? > 9. What is the profile of resource usage, like memory in use? > 10. What is the profile of processor latency? > - flowfiles backed up awaiting processing > - flowfiles in other states (?) > > Any nudge on any of this would be very welcome. NiFi is new enough that > there's precious few samples of people using the ReST interface. > > Thanks > > > > > >
