Thanks Matt, I already have scheme replicated. Will try this method and let you know. If first time transfer works, with QueryDatabaseTable I should be able to replicate new records automatically, which is what I am planing to have. Thanks, -Sumo
Sent from my iPhone > On Aug 12, 2016, at 8:46 AM, Matt Burgess <[email protected]> wrote: > > Sumo, > > One-time migrations are possible with NiFi (although probably not a > common use case). Are the target tables created already? Some DBs > (like MySQL) support queries that return the Create Table statement, > if you get those you can use PutSQL to execute them on your target > system. > > If the target tables are created, you can use QueryDatabaseTable -> > ConvertAvroToJSON -> ConvertJsonToSQL -> PutSQL, the conversions > should leave you with flowfiles containing INSERT statements. If you > already know the schema, you could probably use ReplaceText instead of > ConvertJsonToSQL. > > QueryDatabaseTable [1], if you configure it with a maximum-value > column like the primary key column, will only fetch the data once. It > keeps track of the largest value in the column you specify, so as long > as more data (with higher values in that column) are not added to the > source table, then QueryDatabaseTable can continue to "run" but it > won't output any records after the first full fetch. > > Another option might be to export the source table to a file, then > transfer it with NiFi to the target system, then use PutSQL to do a > BULK INSERT [2]. > > Regards, > Matt > > [1] > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.QueryDatabaseTable/index.html > [2] https://msdn.microsoft.com/en-us/library/ms188365.aspx > >> On Thu, Aug 11, 2016 at 7:09 PM, Sumanth Chinthagunta <[email protected]> >> wrote: >> I have A set of tables to transfer, but as I have small set of tables , I >> can duplicate flows if flow design works with one table. >> Basically migrating an app from mainframe/DB2 to MS SQL server. >> Thanks >> Sumo >> Sent from my iPhone >> >>> On Aug 11, 2016, at 1:17 PM, Jeff <[email protected]> wrote: >>> >>> Hello Sumo, >>> >>> Are you looking at transferring data from a specific table between two >>> databases, or all tables? >>> >>> On Wed, Aug 10, 2016 at 3:17 PM Sumanth Chinthagunta <[email protected]> >>> wrote: >>> >>>> I have same scheme in DB2 and SQL server. >>>> Is there a simple solution to transfer data with NiFi? ( one time ) >>>> Hoping for somebody have better idea to build the flow :) >>>> Thanks >>>> Sumo >>>> Sent from my iPhone >>>>
