That was it! The fragmented transactions settings were not correct. Thanks a million!
On Wed, Aug 15, 2018 at 11:12 AM, Matt Burgess <[email protected]> wrote: > Do you have a Split processor upstream? If so, is the setting of > Support Fragmented Transactions in PutSQL set to true? That > combination will have PutSQL try to find all the flow files with the > same "fragment.id" attribute, and will only proceed if it gets > "fragment.count" of them (all the flow files with the same fragment.id > should have the same fragment.count value, PutSQL checks for this and > should log an error/warning if not). In order to get that to work, the > Batch Size property would have to be at least as big as what you > expect your fragment.count to be (perhaps twice the size). You may > also want to use FirstInFirstOutPrioritizer on all the connections > downstream from the Split. > > Alternatively, if the order of the incoming SQL statements doesn't > matter, set Support Fragmented Transactions to false. > > Having said all that, unless you have a reason to generate your own > SQL, consider using PutDatabaseRecord rather than Split -> Replace -> > PutSQL, it handles all your upstream data as one flow file (no need > for split) and generates the SQL (no need for replace) and executes it > for all the records in the flow file (no need for PutSQL). There are > some reasons why PutDatabaseRecord wouldn't be the right choice; often > it's when you need specific statements or ones that aren't currently > supported (such as UPSERT), or when you're using a different Put than > PutSQL (such as PutHiveQL). Otherwise I always recommend > PutDatabaseRecord when prudent. > > Regards, > Matt > > On Wed, Aug 15, 2018 at 10:44 AM Lone Pine Account <[email protected]> > wrote: > > > > I have a simple flow that takes the output of a ReplaceText processor > and sends it to PutSQL. > > > > This has been working in the past with a "toy" configuration. Now that > I'm testing it on a larger input set, it is not working but I can't figure > out why. > > > > Attached is my setup. The ReplaceText processor fills up the success > queue, and rather than processing these it appears that the PutSQL just > keeps filling up its "In" port with more copies of them - the number of > "In" for PutSQL keeps going up, but nothing comes off the success queue. > > > > So I'm trying to debug: > > - I've attached LogAttribute processors to the ReplaceText processor, > and verified that the SQL commands are correct. > > - I've attached LogAttribute processors to all of the PutSQL > relationships, and none receive output. > > - I've looked through the app logs and there is nothing coming up for my > PutSQL processor. > > > > Where can I look to figure out why the PutSQL processor is no longer > writing to the database? > > > > Thanks >
