There is a collection of templates for common patterns at https://github.com/hortonworks-gallery/nifi-templates
Feel free to submit any interesting use cases as PRs to that GitHub. Simon On 24 Mar 2016, at 17:50, Uwe Geercken <[email protected]<mailto:[email protected]>> wrote: Joe yes I know this one. but with obviously so much interest in nifi there must be lots of good ideas around. can we not collect templates on github somewhere? rgds uwe -- Diese Nachricht wurde von meinem Android Mobiltelefon mit WEB.DE<http://WEB.DE> Mail gesendet. Am 24.03.2016, 18:03, Joe Witt <[email protected]<mailto:[email protected]>> schrieb: Totally agree with you. How about his for now https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates And we can come up with something better (like the registry) in the future. Thanks Joe On Thu, Mar 24, 2016 at 11:00 AM, Uwe Geercken <[email protected]<mailto:[email protected]>> wrote: > I believe it would be cool to have a central place where we can store and > download templates. today so many people do cool stuff but its all > decentralized. > > maybe it is just a matter of deciding where > > uwe > -- > Diese Nachricht wurde von meinem Android Mobiltelefon mit WEB.DE Mail > gesendet. > Am 23.03.2016, 23:57, Russell Bateman > <[email protected]<mailto:[email protected]>> > schrieb: >> >> Hope not to make copy/paste mistakes. I can share the code snippets, but >> unfortunately not a trace. I didn't keep one and my efforts to restore >> the broken code only leave me frustrated. (I've obliterated my NAR >> several times, but the debugger tells me that the lines don't match up >> when I step through, can't set breakpoints, etc. Sorry.) >> >> How I was doing it: >> >> public void onTrigger( final ProcessSession session, final >> DataExtractor filter, final String regexPattern ) >> throws ProcessException >> { >> final FlowFile flowfile = session.get(); >> * final AtomicReference< FlowFile > flowFileHolder = new >> AtomicReference<>();** >> ** flowFileHolder.set( flowfile );* >> >> session.read( flowfile, new InputStreamCallback() >> { >> @Override >> public void process( InputStream in ) throws IOException >> { >> RegularExpressionMatch match; >> >> try >> { >> // slurp the in-coming stream to match into... >> Matcher matcher = Pattern.compile( regexPattern ).matcher( >> StreamUtilities.slurp( in, -1 ) ); >> if( !matcher.find() ) >> throw new IOException( "Failed to match regular expression >> pattern in the document" ); >> match = new RegularExpressionMatch( matcher ); >> } >> catch( Exception e ) >> { >> throw new IOException( "Failed to create regular expression >> matcher with passed properties" ); >> } >> >> Tag tag = filter.buildTag( match ); >> String xml; >> xml = ( Utilities.isEmpty( tag ) ) >> ? "(No tag built for this document.)" >> : new TagUtilities( tag, true /*properties.debug*/ >> ).getXmlFromTag( 0 ); >> * try** >> ** {** >> ** FlowFile flowFileWithAttributes = flowFileHolder.get();** >> ** flowFileWithAttributes = session.putAttribute( >> flowFileWithAttributes, "concept", xml );** >> ** flowFileHolder.set( flowFileWithAttributes );** >> } >> catch( Throwable e ) >> { >> -----> e.printStackTrace(); // IllegalStateException >> } >> * } >> } ); >> >> // the flowfile now has attributes we put on it... >> session.transfer( flowFileHolder.get(), new >> ProcessorRelationships().getSuccess() ); >> } >> >> How I'm doing it now: >> >> public void onTrigger( final ProcessSession session, final >> DataExtractor filter, final String regexPattern ) >> throws ProcessException >> { >> FlowFile flowfile = session.get(); >> * final String attribute;* >> * final AtomicReference< String > attributeHolder = new >> AtomicReference<>();* >> >> session.read( flowfile, new InputStreamCallback() >> { >> @Override >> public void process( InputStream in ) throws IOException >> { >> RegularExpressionMatch match; >> >> try >> { >> // slurp the in-coming stream to match into... >> String incoming = StreamUtilities.slurp( in, -1 ); >> Matcher matcher = Pattern.compile( regexPattern ).matcher( >> incoming ); >> if( !matcher.find() ) >> throw new IOException( "Failed to match regular >> expression pattern in the document" ); >> match = new RegularExpressionMatch( matcher ); >> } >> catch( Exception e ) >> { >> throw new IOException( "Failed to create regular expression >> matcher with passed properties" ); >> } >> >> Tag tag = filter.buildTag( match ); >> String xml; >> xml = ( Utilities.isEmpty( tag ) ) >> ? "(No tag built for this document.)" >> : new TagUtilities( tag, true /*properties.debug*/ >> ).getXmlFromTag( 0 ); >> * attributeHolder.set( xml );* >> } >> } ); >> >> /* Can't change anything about the session right inside here: >> don't change flowfiles >> * in the callback! Set the attribute here instead of inside the >> read! >> */ >> * attribute = attributeHolder.get();* >> * flowfile = session.putAttribute( flowfile, "concept", attribute );* >> >> // the flowfile now has attributes we put on it... >> session.transfer( flowfile, new >> ProcessorRelationships().getSuccess() ); >> } >> >> On 03/23/2016 03:42 PM, Oleg Zhurakousky wrote: >> > Russell >> > >> > This doesn't sound right. Would you care to share a code snippet on how >> > you set the attribute as well as stack trace? >> > >> > Cheers >> > Oleg >> > >> >> On Mar 23, 2016, at 5:33 PM, Russell Bateman >> >> <[email protected]<mailto:[email protected]>> >> >> wrote: >> >> >> >> I stumbled upon something peculiar in NiFi. I had been attaching an >> >> attribute to a flowfile in the session.read() call-back. The NiFi unit >> >> testing framework tolerated this, but when I finally ran my processor for >> >> real, it blew chunks (IllegalStateException). I solved the problem by >> >> saving >> >> the new attribute and attaching it instead outside the call-back just >> >> before >> >> calling session.transfer(). >> >> >> >> The only reason I'm pointing this out is because I somewhat rely on >> >> unit testing and hope to catch this level of error/gotcha earlier than >> >> button-pushing testing >> >> >> >> --in case anyone cares. >> >> >> >> Open to comments, cat-calls, etc. >> >> >> >> ;-) >> >> >> >> Best regards, >> >> >> >> Russ
