Hi Matthew , I didn't read your reply(just now reading :)), however, I have implemented in the same way as you described :)).
In fact, I followed steps in FetchFile processor to read files directly from an input list generated by ListFile processor. My flow looks like below: Flow: An Input Folder > [1]ListFile(Standard Processor) > [2]NzLoad(My Custom processor) > [3]NzSql (My Custom processor) Now need your help on below: 1. As I am reading from ListFile, it should be thread safe (a single file should not be processed by more than 1 threads on NzLoad), right ? 2. How can I make the full flow synchronous. I want it like below: a. [1]ListFile should list <=N files for each run and then stop until [2] & [3]c completes b. [2]NzLoad will process all the <=N files(using multiple thread) and only then then trigger [3]NzSql c. [3]NzSql will complete it processing and then trigger [1]ListFile and so on. Please share me if you have any alternative idea. Regards, Obaid On Fri, Jan 29, 2016 at 4:22 AM, Matthew Burgess <[email protected]> wrote: > I think an approach that could work is the following: > > 1) Start with a ListFile processor to locate the filenames of the files > you want to load into netezza > > 2) Send to a ReplaceText processor, where you replace the entire contents > of the flow file with: > insert into MYTABLE > select * FROM EXTERNAL '${filename}' > USING (<JDBC Options>) ; > > > Note I’m using Expression Language to refer to the “filename” attribute. > Each incoming flow file will have that attribute set to one of the files > found by the ListFile processor. > > > > 3) Send to a PutSQL processor. Your SQL statement(s) are complete at this > point so you don’t need arguments, parameters, etc. > > I haven’t tried this yet but it seems like it should work. If you give it > a try, please let us know how it works for you :) > > Regards, > Matt > > > On 1/28/16, 3:04 PM, "Joe Percivall" <[email protected]> wrote: > > >Hello Obaid, > > > >Sorry no one has gotten back to you sooner, many of the developers are > working diligently to get 0.5.0 done. > > > >I don't know too much about loading SQL tables but when you say "local > file system filename", is this a file that exits on the target system? If > so you may just be able to set that command because my understanding is > that using ExecuteSQL you have an input query which gets sent to the target > server. Then the target server then runs that command. > > > >Can anyone else that has more experience with SQL and loading tables > chime in? > > > > > >Joe > >- - - - - - > >Joseph Percivall > >linkedin.com/in/Percivall > >e: [email protected] > > > > > > > >On Saturday, January 23, 2016 12:31 AM, obaidul karim < > [email protected]> wrote: > > > > > > > >Hi, > > > >I developing a nifi processor for netezza. > >As per by experience the best way to load data to netezza is using > netezza JDBC external table. > > > >I want to run below command to load a file within NiFi processor. > > > >insert into MYTABLE > >select * FROM EXTERNAL '<local file system filename>' > >USING (<JDBC Options>) ; > > > >My question is which file to usefor <local file system filename> ? > >- Is it the flow files ? If yes then how can I get full path of a > flowfile ? > >- or can I directly load the file in a spool directory ? > > > > > >Thanks for your help in advance. > > > >-Obaid > >
