Hi Matt. I’ll have a look at option a. What’s the timeframe for a fix and prod release for this assuming it’s a bug? Ballpark.
Thanks, Bryan From: Matt Burgess [mailto:[email protected]] Sent: Monday 24 July 2017 14:40 To: [email protected] Subject: Re: Issue with GenerateTableFetch bulk load when using MS SQL Server database type Bryan, This is likely a bug, I will investigate and write it up if so. In the meantime, are you planning on doing the bulk fetch in parallel (with a Remote Process Group into ExecuteSQL across a NiFi cluster)? If not, you may find that QueryDatabaseTable is a good alternative, it can be configured to fetch N rows at a time. The trade off is that it is not parallel or concurrent (each N rows will be fetched one batch at a time), but then each batch can be processed concurrently or in parallel. Another alternative, if you know the approximate size of the database, is to use GenerateFlowFile to write the SQL statements yourself, then SplitText to get one statement per flow file, then ExecuteSQL (concurrently or in parallel). This solution does not scale with your database size but with some copy and paste (and updating as the DB grows), can emulate GenerateTableFetch in the short term. Regards, Matt On Jul 24, 2017, at 8:55 AM, Bryan Quinn <[email protected]<mailto:[email protected]>> wrote: Hi, I’m trying to use the GenerateTableFetch component for ‘MS SQL 2012+’ database – for a full table export each time its run. However I’m forced to set a column name in the ‘Maximum-value Columns’ field when the component is run (error only shows during execution if I omit this setting). I’m seeing the following error: IllegalArgumentException: Order by clause cannot be null or empty when using row paging So it looks like it only works with incremental mode – which I don’t want. Can someone see why this is the case or is it a bug? I’m not getting this error if I set the database type to ‘Generic’ – but unfortunately the sql it generates isn’t valid for MS SQL server. I’m using Nifi v1.3.0 Thanks, Bryan --- Bryan Quinn | Asavie Principal Analytics Architect E: [email protected]<mailto:[email protected]> W: www.asavie.com<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Foutlook.office.com%2Fowa%2Fwww.asavie.com&data=02%7C01%7CBryan.Quinn%40asavie.com%7C618b8582903b4775a3f108d4d299870d%7Cff38a05d2e5248389b077fbd50bbae9f%7C0%7C0%7C636365004244466091&sdata=02nM2H4yXmxB4rpeCQGkQaUSec18VB%2FJqoffqekjp74%3D&reserved=0> | T: @asavie
