Do you have a Split processor upstream? If so, is the setting of Support Fragmented Transactions in PutSQL set to true? That combination will have PutSQL try to find all the flow files with the same "fragment.id" attribute, and will only proceed if it gets "fragment.count" of them (all the flow files with the same fragment.id should have the same fragment.count value, PutSQL checks for this and should log an error/warning if not). In order to get that to work, the Batch Size property would have to be at least as big as what you expect your fragment.count to be (perhaps twice the size). You may also want to use FirstInFirstOutPrioritizer on all the connections downstream from the Split.
Alternatively, if the order of the incoming SQL statements doesn't matter, set Support Fragmented Transactions to false. Having said all that, unless you have a reason to generate your own SQL, consider using PutDatabaseRecord rather than Split -> Replace -> PutSQL, it handles all your upstream data as one flow file (no need for split) and generates the SQL (no need for replace) and executes it for all the records in the flow file (no need for PutSQL). There are some reasons why PutDatabaseRecord wouldn't be the right choice; often it's when you need specific statements or ones that aren't currently supported (such as UPSERT), or when you're using a different Put than PutSQL (such as PutHiveQL). Otherwise I always recommend PutDatabaseRecord when prudent. Regards, Matt On Wed, Aug 15, 2018 at 10:44 AM Lone Pine Account <[email protected]> wrote: > > I have a simple flow that takes the output of a ReplaceText processor and > sends it to PutSQL. > > This has been working in the past with a "toy" configuration. Now that I'm > testing it on a larger input set, it is not working but I can't figure out > why. > > Attached is my setup. The ReplaceText processor fills up the success queue, > and rather than processing these it appears that the PutSQL just keeps > filling up its "In" port with more copies of them - the number of "In" for > PutSQL keeps going up, but nothing comes off the success queue. > > So I'm trying to debug: > - I've attached LogAttribute processors to the ReplaceText processor, and > verified that the SQL commands are correct. > - I've attached LogAttribute processors to all of the PutSQL relationships, > and none receive output. > - I've looked through the app logs and there is nothing coming up for my > PutSQL processor. > > Where can I look to figure out why the PutSQL processor is no longer writing > to the database? > > Thanks
