We've done something similar as Carlos outlines here. Works really great Dano
On Sun, Aug 13, 2017, 9:39 AM Carlos Manuel Fernandes (DSI) < [email protected]> wrote: > Hi Vikram, > > > > I had the same requirements like you , and my solution is a service: > > > > HandleHttpRequest (prepared to handle > http://host:8085/ods?sourceTable=tableName&targetTable=tableName&truncate=Y) > -> > > ExecuteScript (Script reads the http parameters and/or read extra > parameters from a database table and make the synchronization) -> > > HandleHttpResponse (return 200 if ok ) > > > > After this service are ready, you can create a scheduled invoker , like > this: > > > > ExecuteScript (get all the tables names you need to synchronize, in my > case based on a query : select source_table,target_Table,,truncate from > ods_tables ) -> > > invokeHttp ( > http://host:8085/ods?sourceTable=${source_table}&targetTable=${target_table}&truncate=${truncate}) > -> > > putEmail (sending the result of the synchronization). > > > > > > With this you don’t need to repeat flows, I hope this help. > > > > Carlos Fernandes > > > > > > > > > > *From:* Andy LoPresto [mailto:[email protected]] > *Sent:* sábado, 12 de agosto de 2017 01:36 > *To:* [email protected] > *Subject:* Re: Parameterizing the nifi flow > > > > The variable registry is a great tool for parameterizing values that > differ between environments/deployments. In this case it sounds like > setting up a flow that read from a flowfile attribute/content to determine > the source table’s name is a better fit. You can create a master list of > all source tables and store it as a plaintext file, then on your schedule, > read the contents of that file, split the content by line, and send each > 100+ resulting flowfiles into the part of the flow that reads from the > database. Each would provide the name of the source table to read at that > time. > > > > Andy LoPresto > > [email protected] > > *[email protected] <[email protected]>* > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > On Aug 11, 2017, at 3:26 PM, Andrew Grande <[email protected]> wrote: > > > > Hi, > > Read up on the variable registry in the docs, that sounds like a good fit. > I don't remember if it were available in 1.1 though. > > Andrew > > > > On Fri, Aug 11, 2017, 5:12 PM More, Vikram (CONT) < > [email protected]> wrote: > > Hi, > > > > I have a nifi flow which pulls/extracts from source database table and > loads into target database table. This flow will run several times in a day > to get delta records from source table (more like batch process running > every 3-4 hrs). Now I need to replicate this same process for 100+ > different source tables. So rather than creating 100+ nifi flows for each > separate table, can I create main flow (let's say template) and pass > parameter like source extract sql, target load sql to main flow. And repeat > these steps for each source table . Has anyone tried parameterizing the > nifi flows, can you please advice . We are using NiFi 1.1.0 > > > > Appreciate any thoughts here. > > > > > > Thanks & Regards, > > *Vikram* > > > > > ------------------------------ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. > > >
