RecordReader and RecordWriter: development work on ruleengine

Hello, I would like to pick up development on my processor that uses a ruleengine. It is working since several month already but I want to convert the processor to use the new RecordReader/RecordWriter framework. This way I can use any data format, which is really, really cool and will

Aw: Re: Interest in a time series simulator processor for NiFi

Just my two cents... I have created a processor that generates CSV data.   You can see an example here: http://datamelt.weebly.com/blog/nifi-processor-generatedata   It generates data based on word lists, regular expressions or purely random. Date data can be generated in a way that the columns

Aw: Re: 4 Processors for NiFi: SplitToAttribute, MergeTemplate, ExecuteRuleEngine, GenerateData

record at a time. Their are record readers already for CSV, JSON, AVRO, and Grok with writers in CSV, Avro, JSON, and generic text. Thanks Joe On Fri, Apr 14, 2017 at 11:16 AM, Uwe Geercken <uwe.geerc...@web.de> wrote: > Hello everybody, > > I have released (Apache License) my NiFi proces

Aw: Re: InputStream and LineIterator

single large set of rows and putting out many individual rows? Could you have it put out rows grouped together by some logic or must you end up with individual splits? Thanks Joe On Fri, Apr 14, 2017 at 11:14 AM, Uwe Geercken <uwe.geerc...@web.de> wrote: > Hi everybody, > > in my Exe

4 Processors for NiFi: SplitToAttribute, MergeTemplate, ExecuteRuleEngine, GenerateData

Hello everybody,   I have released (Apache License) my NiFi processors at:   https://github.com/uwegeercken/nifi_processors[https://deref-web-02.de/mail/client/WDRw2DECNaw/dereferrer/?redirectUrl=https%3A%2F%2Fgithub.com%2Fuwegeercken%2Fnifi_processors]   Further below is a summary for the

Processor logic

Hello, I have a little bit of a hard time to design processors correctly. I find it difficult to decide if a processor should e.g. process a single line from a flow file or process also flow files with multiples lines of data (e.g. in the case of CSV files). Another point is the handling of

Aw: Re: Nifi processor documentation not showing up in Help

enu -> right side drop down -> Help)? > > Best Regards, > Uwe > > Am 24.02.2017 um 12:35 schrieb Uwe Geercken: > > Uwe, > > > > wenn man in Nifi die Help aufruft, dann erscheinen alle Hilfstexte für die > > Prozessoren. Nifi genriert diese beim

Central processor repository

Hello, I remember a while ago that there was a short discussion if it would be good to have a central place somewhere where people could upload the processors they created and others could download them from this central point. It would make life easier compared to surfing the web to find them

Nifi processor documentation not showing up in Help

Hello, I am developing some processors for Nifi. They do work in 1.1 but the documentation does not show up in the Help. So I set the loglevel to DEBUG and this is what came out: 2017-02-24 11:00:18,538 DEBUG [main] org.apache.nifi.nar.NarUnpacker Expanding NAR file:

Aw: Re: CsvToAttributes processor

care to document its intended use and limitations on performance to > consider, and enable it to limit how many columns/fields get turned > into attributes maybe by setting a max or by having a > whitelist/blacklist type model. Even if it won't achieve highest > sustained perform

Aw: CsvToAttributes processor

Francois, very nice. Thanks. I have been working on a simple version a while ago. But it had another scope: I wnated to have a Nifi processor to merge CSV data with a template from a template engine (e.g. Apache Velocity). I will review my code and have a look at your processor. Where can we

Aw: Re: [ANNOUNCE] New Apache NiFi PMC Member - Matt Burgess

Congratulations Matt. That's very good for Nifi. I have met Matt in person and he is smart, always helpful and has great expertise. A great person to work with and also to have a beer with ;-) Uwe > Gesendet: Freitag, 07. Oktober 2016 um 18:52 Uhr > Von: "Joe Skora" > An:

Aw: Re: Site-to-site not working

working > > The socket host should be the ip of the machine it is on, not the machine > that it expects to receive the data from. That variable is similar to the > httphost or https host variable, also in the prop file. > > Hope that helps! > > Sent from my iPhone > &

Site-to-site not working

Hello, I have used Nifi for a while on my laptop. Now I wanted to setup a second machine communicating with the first one - either using Nifi oder Minifi. For the moment I have Nifi 1.0.0 running on both and I followed the instructions of how to setup Nifi. I read documentation, but there must

Apache NiFi - Dev and Prod

Hello, are there any recommendations or best practices available on how to setup a dev and prod environment with Apache Nifi? It the simple answer to run Nifi twise on different servers? And if so how would a deployment from dev to prod look like? Through the use of templates? Thanks for any

Aw: Re: Re: Filtering large CSV files

; Hi Uwe, > > Yes, that is what I was thinking of using for the CSV processor. Will you > be committing your version? > > - Dmitry > > On Tue, Apr 5, 2016 at 1:39 PM, Uwe Geercken <uwe.geerc...@web.de> wrote: > > > Dimitry, > > > > I was working

Aw: Re: Filtering large CSV files

Dimitry, I was working on a processor for CSV files and one remark came up that we might want to use the opencsv library for parsing the file. Here is the link: http://opencsv.sourceforge.net/ Greetings, Uwe > Gesendet: Dienstag, 05. April 2016 um 13:00 Uhr > Von: "Dmitry Goldenberg"

Aw: Re: Re: Re: Re: Re: Processor additional documentation

can import the top level POM into your IDE of choice :) > > Regards, > Matt > > > On Mar 21, 2016, at 1:21 PM, Uwe Geercken <uwe.geerc...@web.de> wrote: > > > > I am sorry for that. I have completely overlooked that. > > > > So tell me Matt how is

Aw: Re: Re: Re: Re: Processor additional documentation

Uwe > > > > Noticed your other threads on great progress. That is awesome. > > > > Really want to help you get to the bottom of the additional details > > piece though. We clearly have to do a better job with documenting (or > > implementing) how to do t

Aw: Re: Re: Re: Processor: User friendly vs system friendly design

e may not be safely > published. And the name of the processor should probably reference > Velocity. > > Thanks, > > Adam > > [1] https://issues.apache.org/jira/browse/NIFI > [1] http://opencsv.sourceforge.net > > > > On Sat, Mar 19, 2016 at 2:09

Processor additional documentation

Hello, I am writing my first processor. As described in the documentation, I have added an HTML file to be used when the user selects "Usage": docs/com.datamelt.nifi.test.TemplateProcessor/additionalDetails.html This is located in the root or the Processors nar file. The processor class is

Aw: Re: Re: Processor: User friendly vs system friendly design

utput) > > It's the "​split row into fields and merge with template" that we would > want to separate into two processors instead of one. > > You're very much on the right track, I believe. If the above doesn't help, > I'll try and jump in on a code example when I

Re: Processor: User friendly vs system friendly design

ng your Velocity template transform using flowfile attributes as opposed to CSV? Flowfile attributes are NIFI's lowest common data representation, many many processors create attributes which would enable your Velocity processor to be used by more than just CSV input. Adam On Fri, Mar 18, 2016

Processor: User friendly vs system friendly design

Hello, my first mailing here. I am a Java developer, using Apache Velocity, Drill, Tomcat, Ant, Pentaho ETL, MongoDb, Mysql and more and I am very much a data guy. I have used Nifi for a while now and started yesterday of coding my first processor. I basically do it to widen my knowledge and

Aw: Re: Processor: User friendly vs system friendly design

be really nice. > Extra bonus points for Markdown or Asciidoc transforms as well (but these > might be too separate of a use case). > > Hope this helps. > > Adam > > [1] http://nifi.apache.org/developer-guide.html > > [2] https://cwiki.apache.org/confluence/display/NIFI/Contribu

Aw: Re: Processor additional documentation

nfluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExtensions-ExampleProcessorBundleStructure] If you think that part is setup correctly, can you check under nifi_home/work/docs and see if com.datamelt.nifi.test.TemplateProcessor is there? -Bryan On Fri, Mar 18, 2016 at 11:04

Aw: Re: Re: Re: Processor additional documentation

s://nifi.apache.org/docs.html > > Dan > > On Fri, Mar 18, 2016 at 11:08 AM Uwe Geercken <uwe.geerc...@web.de> wrote: > > > Dan, > > > > but maybe I have a wrong understanding: do I have to create an index.html > > file? Currently I have only